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Foreword 


The mission of the International Association for the Evaluation of Educational 
Achievement (IEA) is to enhance knowledge about education systems worldwide 
and to provide high-quality data that will support education reform and lead to 
better teaching and learning in schools. In pursuit of this aim, it conducts, and 
reports on, major studies of student achievement in literacy, mathematics, science, 
citizenship, and digital literacy. These studies, most notably the Trends in 
International Mathematics and Science Study (TIMSS), Progress in International 
Reading and Literacy Study (PIRLS), and International Civic and Citizenship 
Education Study (ICCS), are well established, and have set the benchmark for 
international comparative studies in education. 

The studies have generated vast datasets encompassing student achievement, 
disaggregated in a variety of ways, along with a wealth of contextual information 
which contains considerable explanatory power. The numerous reports that have 
emerged from them are a valuable contribution to the corpus of educational 
research. 

Valuable though these detailed reports are, IEA’s goal of supporting education 
reform needs something more: deep understanding of education systems and the 
many factors that bear on student learning requires in-depth analysis of the global 
datasets. IEA has long championed such analysis, and facilitates scholars and policy 
makers in conducting secondary analysis of our datasets. So we provide software 
such as the International Database Analyzer to encourage the analysis of our 
datasets, support numerous publications including a peer-reviewed journal— 
Large-scale Assessment in Education—dedicated to the science of large-scale 
assessments and publishing articles that draw on large-scale assessment databases, 
and organize a biennial international research conference to nurture exchanges 
between researchers working with IEA data. 

This new series of thematic reports represents a further effort by IEA to capi- 
talize on our unique datasets, so as to provide powerful information for policy 
makers and researchers. Each report will focus on a specific topic and will be 
produced by a dedicated team of leading scholars on the theme in question. Teams 
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are selected on the basis of an open call for tenders. The intention is to have two 
such calls a year. Tenders are subject to a thorough review process, as are the 
reports produced. (Full details are available on the IEA website.) 

This first report is based on secondary analysis of the Progress in International 
Reading Literacy Study (PIRLS) data, and aims to deepen understanding of the 
influence of home support for learning on student achievement. Conducted by 
researchers at the University of Twente, Psychometric Framework for Modeling 
Parental Involvement and Reading Literacy is a valuable addition to the growing 
research on the impact of parental involvement in education, exploring the rela- 
tionship between aspects of parental involvement and student reading literacy 
across countries, while taking cultural differences into account. This report should 
make a significant contribution to a field where cross-cultural comparisons from a 
triangulated perspective are sparse. 

Upcoming thematic reports will focus on the role of teachers and teaching in 
relationship to student outcomes, and on perceptions of school safety and the school 
environment for learning and their impact on student learning. 


Seamus Hegarty 
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Executive Summary 


Research Objectives 


Parental involvement is seen as one of the most malleable factors of the student's 
home situation, which makes it a relevant subject for schools, educational policies, 
and research. Though many studies have researched its role in student achievement, 
effects are not univocal. It is difficult to tell whether these inconsistent results are 
caused by differences between educational systems and cultural differences, or by 
the great variation in the methods used to assess student achievement and parental 
involvement across studies. Large-scale international assessment studies like the 
Progress in International Reading and Literacy Study (PIRLS) undertaken by the 
International Association for the Evaluation of Educational Achievement 
(IEA) therefore provide valuable data. However, a limiting factor may be the extent 
to which data from different countries can be compared, as cultural differences 
could influence the international validity of the indicators. 

The purpose of this research was to develop a suitable psychometric framework 
for the assessment of the relation between parental involvement and reading liter- 
acy, using the PIRLS-2011 data for 41 countries as a case study. The framework 
included country-specific differences, both at the item and scale level, to gain 
insight into cultural differences in the parental involvement construct and its rela- 
tion to student achievement in reading literacy. The project may thus also contribute 
to the issue of measurement invariance across cultures in large-scale assessment 
studies. 

Three main research questions guided the project: 


e Which dimensions of parental involvement can be discerned and to what extent 
is there empirical evidence that these dimensions are related to student 
attainment? 

e To what extent are there any cultural differences (differences between countries) 
in the constructs that measure dimensions of parental involvement? 


ix 
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e To what extent are the different dimensions of parental involvement related to 
student achievement in reading literacy, taking into account student background 
characteristics and differences between countries? 


Literature Review 


The introductory literature review (1) Investigated the underlying dimensions of the 
parental involvement concept in current research literature and how these dimen- 
sions were related to educational outcomes; (2) explored which dimensions showed 
the most significant relationship with student attainment, and reading literacy in 
particular; and (3) determined whether cultural differences existed between coun- 
tries in the perception of relationships between parental involvement and student 
achievement. 

The review identified four key dimensions of parental involvement and provided 
a comprehensive analysis of existing studies on parental involvement and student 
achievement. Overall, the literature suggested that parental involvement had a 
positive influence on student achievement; however, when the key four dimensions 
characterizing parental involvement were considered separately, the results were 
less straightforward. None of the meta-studies in the literature addressed cultural 
differences in parental perceptions of parental involvement or how parents behaved 
towards their child’s education as possible reasons for the diverse results. 


Modeling Parental Involvement 


The psychometric framework aimed to identify and model cultural differential item 
functioning (CDIF) using five item response theory (IRT) models: the generalized 
partial credit model (GPCM), a GPCM with 10 % and 20 % country-specific 
parameters, a GPCM with random item parameters, and the bi-factor GPCM. These 
models were each applied to five components of parental involvement identified by 
the literature review and constructed from the PIRLS-2011 dataset. 

All models clearly and consistently supported the identification of CDIF. 
However, the results obtained varied across models. This disagreement was because 
different aspects of model fit were assessed by the models; the method using 
residuals (i.e., the first four models) specifically targeted uniform CDIF, while the 
bi-factor GPCM specifically targeted nonuniform CDIF. Practitioners should not 
rely on one model and one approach to investigate CDIF, but diversify in their 
methods. 
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Latent Multilevel Model 


After modeling parental involvement including CDIF, its relation with reading 
literacy was investigated using a latent three-level regression model. The analyses 
confirmed that the control variables for gender and socioeconomic status 
(SES) were important predictors for reading literacy. Parental involvement in early 
literacy activities with their child had a small but positive influence on student 
achievement. Parental involvement in helping their child with homework in the 
current grade appeared to have a weak negative influence. The effect of other 
components was negligible. 

A model with random slopes at the country level for the components “early 
literacy activities” and “helping with homework” was tested to establish whether 
their effects differed across countries. There were large differences in the impact 
across countries, especially for parental help with homework. There was a positive 
correlation between the country-level intercept slopes, indicating that the effect of 
helping with homework was less significant in low-achieving PIRLS countries than 
in high-achieving countries. 

To establish the impact of CDIF, the analysis was replicated with the a posteriori 
estimates of the latent student parameters from all five IRT models. The resulting 
estimates were very close, and within one standard deviation from the estimates 
under the GPCM. Thus, CDIF did not appear to bias the inferences. 


Conclusions 


The main purpose of this study was to assess whether there were cultural differences 
in the measurement of parental involvement in PIRLS and, if so, whether correction 
for these differences led to differing interpretations with regard to its relation with 
reading literacy. Although some of the constructed scales for parental involvement 
may require improvements to increase reliability, the PIRLS scales remain inter- 
nationally valid. Analyzing the impact of CDIF on the estimates of country means 
and on the outcomes of latent regression analyses established that CDIF did not 
influence the results. 

By providing a robust framework to identify and model cultural differences, this 
project contributes to the field of large-scale assessments in educational research by 
modeling the relationships between parental involvement and student learning over 
a large number of educational systems. Most importantly, the work confirms that 
there is a positive relationship between parents’ early literacy activities and stu- 
dents’ reading literacy, and conversely, a negative relationship between parental 
help with homework and students’ reading literacy. 


Chapter 1 
Introduction 


Although the role of parental involvement in student achievement (and reading 
literacy in particular) is widely acknowledged, research findings regarding its effect 
differ considerably (Fan and Chen 2001; Hoover-Dempsey et al. 2001; Mattingly 
et al. 2002; Jeynes 2005; Patall et al. 2008). Owing to the large variation in the 
methods used to measure student achievement and parental involvement in these 
studies, it is difficult to establish whether the inconsistency in the results is caused 
by differences between educational systems and cultures, or by the method applied 
and the instruments used. Empirical research is required into measurement of stu- 
dent achievement and indicators of the parental involvement, and how comparisons 
can be made between educational systems (countries), to find out to what extent and 
under which conditions, parental involvement influences student achievement. 
In-depth analyses of large-scale international comparative data, such as that con- 
tained in the Progress in International Reading and Literacy Study (PIRLS) 
undertaken by the International Association for the Evaluation of Educational 
Achievement (IEA), may provide valuable additions to the research into parental 
involvement. 

The main purpose of this research was to develop a psychometric framework for 
assessing the relation between parental involvement and reading literacy, using the 
PIRLS-2011 data as a case study. The framework included country specific dif- 
ferences, both at the item and scale level, to gain insight into cultural differences in 
the parental involvement construct and its relation to student achievement in 
reading literacy. One of the possible limitations of international large-scale 
assessment studies such as PIRLS is the extent to which the data of different 
countries can be usefully compared. Despite the high quality demands for the 
translation of the instruments and the conditions of administration in each partici- 
pating country, cultural differences could influence the international validity of the 
indicators measured. PIRLS offers international scales based on partial credit item 
response theory (IRT) models, which can improve the quality of secondary analyses 
of data. However, the differences in the Cronbach’s alphas of scales reported in the 
PIRLS-2011 international report suggest that the meaning of a scale and the 
interpretation of items within a scale could differ between countries (Martin and 
Mullis 2012). 
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2 1 Introduction 


The main research objective of this study was to assess the extent to which 
parental involvement is related to the achievement of primary school students in 
reading literacy, taking into account student background characteristics, differences 
between schools and countries, and cultural differences in the parental involvement 
construct. More specifically, the study aimed to respond to the following research 
questions: 


(1) Which dimensions of parental involvement can be discerned and to what 
extent is there empirical evidence that these dimensions are related to student 
attainment? 


To answer this first question, we reviewed the recent research literature on parental 
involvement. The main goal was to identify dimensions of parental involvement 
and to gain further insight into the extent to which the different dimensions were 
potentially related to student achievement in reading literacy and other 
achievement-related outcomes. 


(2) To what extent are there any cultural differences (differences between coun- 
tries) in the components that measure dimensions of parental involvement? 


To address this question, five extracted item sets using IRT models were studied for 
item-by-country interactions in item parameters, indicating cultural differential item 
functioning (CDIF). The five scales were subsequently modeled using random and 
country-specific item parameters for the most extreme interactions. A bi-factor IRT 
model was applied, where the country-specific covariance matrix gives an indica- 
tion of the extent to which the scale loads on the intended latent variable and the 
extent to which the responses are country-specific. 


(3) To what extent are the different dimensions of parental involvement related to 
student achievement in reading literacy, taking into account student back- 
ground characteristics and differences between countries? 


A multilevel analysis of the PIRLS-2011 datasets from 41 countries explored the 
relationship between parental involvement and student reading literacy. For all 41 
countries, data from students, schools and parents were available. The analyses used 
a three-level model (consisting of a student, school and country level). The third 
question was addressed by exploring the three-level model with a random intercept 
with fixed effects for the predictors and by exploring a random model for each 
component showing a meaningful relation with achievement in the random inter- 
cept model. This random model explores the extent to which the association 
between the dimensions of parental involvement and student reading literacy differ 
between countries. Subsequently, a model without a correction for country differ- 
ences in the parental components was compared with models with different types of 
corrections for country differences in the parental components. 

In Chap. 2, we review the current literature on parental involvement and its 
perceived influence on student achievement. Chapter 3 covers the measurement of 
parental involvement in the PIRLS background questionnaires. In the first part of 
Chap. 4, we present a psychometric framework to identify and model CDIF in 
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multiple ways, including a non-standard application of the bi-factor model. We 
discuss the results of all approaches for each of the five parental involvement 
components in the second part of the chapter, providing insights into the extent to 
which they are affected by CDIF. The last step in the psychometric framework was 
to relate the parental involvement components to reading literacy, and compare the 
outcomes for the different methods used to model CDIF. In Chap. 5, we present the 
method and the multilevel analyses. Finally, in Chap. 6, we evaluate the meaning of 
the results both for measuring parental involvement in an international context, and 
in discussions of the importance of parental involvement for student attainment. 
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Chapter 2 
Literature Review 


2.1 Introduction 


Parental involvement, that is “parents' interactions with schools and with their 
children to promote academic success” (Hill and Taylor 2004, p. 1491), is an 
umbrella term that includes a variety of behaviors and activities of parents directly 
or indirectly related to the education of their children. Therefore, the review of the 
literature had three main goals. First, to investigate the underlying (sub)dimensions 
of the parental involvement concept in the research literature and theories about 
how these dimensions of parental involvement are related to educational outcomes. 
Second, based on empirical studies, to explore which of these dimensions of par- 
ental involvement show the most potential to be related to student attainment, and 
with reading literacy in particular. The third aim of the review was to identify any 
indication of cultural differences between countries in how parental involvement is 
perceived and its relation with student attainment. 

A literature search was conducted during the spring of 2015 using the social 
research databases ERIC, Web of Science, Scopus and PsycINFO. The search keys 
included (combinations of) parental involvement, parents, parental support, 
parent-child relationship, parent-school partnership, home environment, student 
achievement, student performance, reading performance, and reading literacy. 

In reviewing the literature, we did not aim to present an exhaustive overview of 
all published studies about this subject. To better explore the relationship between 
dimensions of parental involvement and student attainment (the second goal of this 
review), we targeted our search by applying a number of exclusion and inclusion 
criteria. Non-empirical studies and empirical studies that were published before 
2000 were excluded. However, to obtain comprehensive knowledge of the (sub) 
dimensions underlying parental involvement and incorporate theories about how 
these dimensions of parental involvement are related to educational outcomes, we 
included theory-based publications and some older publications. 
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6 2 Literature Review 


Because a systematic quality analysis of the published studies was not part of 
this review, only studies published in peer-reviewed journals were included, to 
ensure a certain methodological rigor. As PIRLS investigates the reading literacy of 
grade 4 students, and research has indicated that parental involvement and its effects 
change as children become older (Domina 2005; Mattingly et al. 2002), only 
studies focusing on primary school students were included. Studies focusing on 
special groups (e.g., students from minority groups, dyslexic students, and urban 
students) were also excluded, as this study does not focus on these special groups 
and comparisons between them across countries are difficult. Furthermore, we only 
included studies that took the effects of variables related to socioeconomic status 
(SES) at family or school level into account. Finally, although this part of the 
review focused mainly on studies investigating the relationship between parental 
involvement and students’ reading literacy, studies using other achievement-related 
outcomes (such as math achievement, grade retention, and motivation) were also 
included. 

Because of the abundance of studies on this subject, several meta-studies 
(meta-reviews or meta-analyses) have been published. We used the same exclusion 
and inclusion criteria for these studies. However, in this case, studies focusing 
(also) on secondary school students were included, since most of the available 
meta-studies included these students in their review. 

For the second part of the review, 22 articles were summarized, including nine 
meta-studies. We present these studies within a general framework for parental 
involvement (Sect. 2.4), based on the information derived from the primary liter- 
ature review (see Sect. 2.3). 


2.2 Theories About the Importance of Parental 
Involvement for Student Attainment 


As parental involvement is one of the most malleable factors of the student’s home 
situation, it has become a relevant subject for schools and educational policy. 
Encouraged by studies showing a positive relationship between parental involve- 
ment and student attainment, educational policy makers in the USA, and in western 
European countries such as the UK and the Netherlands, have initiated and stim- 
ulated parent participation programs over the last 15 years (Mattingly et al. 2002; 
Driessen et al. 2005; Bakker et al. 2007; Patall et al. 2008; Selwyn et al. 2011). 
Schools are increasingly using digital technologies to support and promote the 
involvement of parents, by providing laptop computers, online intranets or learning 
platforms (Selwyn et al. 2011). This encouragement is sometimes less voluntary; in 
the UK, some schools present parents mandatory school-home agreements about 
their involvement in homework and expected behavior, such as attending parent 
meetings (Selwyn et al. 2011). 
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The underlying assumption of these initiatives is that parental involvement and 
educational outcomes are not only correlated, but that parental involvement actually 
influences educational outcomes (Mattingly et al. 2002). In the studies we 
reviewed, we found theoretical discussions concerning the influence of parental 
involvement on student achievement that provided conditions under which students 
may profit from this involvement. Hoover-Dempsey and Sandler (1997) developed 
a model summarizing the factors influencing parental involvement at five levels: 


(1) the decision to become involved (e.g., school invitations, parent role 
construction); 

(2) the choice of type of involvement (e.g., parents’ skills, knowledge, and 
available time); 

(3) how involvement influences school outcomes (e.g., modeling, reinforcement, 
instruction); 

(4) tempering or mediating variables (e.g., fit between parents’ involvement 
actions and school expectations); and 

(5) student outcomes (e.g., achievement, self-efficacy). 


How parental involvement influences school outcomes, namely the third level in 
Hoover-Dempsey and Sandler’s (1997) model, was the main focus of McNeal 
(1999). McNeal (1999) argued that the involvement of parents in their child’s 
education influences student outcomes through three mechanisms. The first 
mechanism, socializing, refers to home-based involvement, such as supervising 
homework, by which parents emphasize the importance of schooling. Generating 
social control through school-based involvement is the second mechanism; 
school-based involvement offers parents the opportunity to develop relationships 
with teachers and other parents, and in discussing their child’s behavior, to learn 
from them. The last mechanism is having access to insider information by com- 
municating with the school. By this mechanism, parents will be, for example, 
earlier and better informed about the available solutions in instances of learning or 
behavioral problems. McNeal’s theory specifies different outcomes for the three 
mechanisms: socialization and social control affect the attitude, motivation and 
behavior of the student, while having access to insider information has a direct 
effect on both cognitive and behavioral student outcomes (McNeal 1999). 

Generating social control and having access to insider information can also be 
regarded as part of a mechanism called increasing social capital (Hill and Taylor 
2004). By being engaged in school-related activities, parents learn about schools’ 
expectations of student behavior and homework and learn how to help with their 
child’s homework and learning at home. They also learn from other parents about 
available and suitable options to improve their child’s learning. At the same time, 
teachers learn about the parental expectations of their students. 

Grolnick and Ryan’s (1989) theory on “parenting styles” focused on the effects 
of home-based parental involvement. According to this theory, three dimensions of 
parenting style are important for the learning behavior and achievement of students. 
The first is support for autonomy, by which children are encouraged in independent 
problem solving and participation in decision making. The second style, direct 
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parental involvement, refers to the extent to which parents are interested in, 
knowledgeable about, and taking an active part in their child’s life. The third 
parenting style that may have a positive influence on student attainment is provision 
of structure (i.e., providing clear and consistent guidelines and rules) with regard to 
homework or other school-related aspects. Cooper et al. (2000) applied this theory 
in their study on parental involvement in homework and showed (1) a positive 
relationship between support for autonomy and student achievement, and (2) a 
negative association between direct parental involvement and student achievement. 
These findings are explained by the student’s performance at school; parents of 
low-performing students become more proactively involved with their child’s 
learning, while parents of students doing well at school, are more likely to support 
the autonomy of their child. 

With regard to the role of parental support for homework, it is widely assumed 
that when parents help their child with homework there are positive effects on 
student learning and academic achievement; the student will study more efficiently, 
effectively and with greater focus (Hoover-Dempsey et al. 2001; Patall et al. 2008). 
In the long term, parental involvement may have a positive effect on student’s 
ability to engage in adaptive self-regulation by promoting the development of 
learning strategies such as goal-setting, planning, time management, and atten- 
tiveness (Patall et al. 2008). Hoover-Dempsey et al. (2001) described three reasons 
why helping with homework may have these positive effects: (1) modeling, 
(2) reinforcement, and (3) instruction (see also their general model for parental 
involvement, mentioned previously). First, while helping their child with home- 
work, parents can serve as salient models. This is based on the idea that children 
learn through observation. Parents are influential role models because they possess 
skills and abilities that children value highly. Because there are no direct conse- 
quences of the child's performance at home (in contrast to school), home provides a 
safe environment where the parent becomes an even more powerful role model. The 
second reason is reinforcement; by providing positive consequences in response to 
the child's homework behaviors, the child is stimulated to demonstrate similar skills 
and behaviors again. Parents may even have an advantage over teachers because 
they have better insight into which reinforcement strategies are the most effective 
for their child. Finally, helping with homework may have a positive effect on 
student achievement because parents tend to use the learning strategy "guided or 
collaborative learning" (Hoover-Dempsey et al. 2001). This includes directing the 
child to the task at hand, simplifying the task, providing extra explanation, or 
relating the task to familiar contexts. 

However, some scholars argue that parental involvement with homework may 
also have negative effects (Patall et al. 2008; Dumont et al. 2012). They point out 
that the involvement may lead to tensions between parents and children, caused by 
parental frustrations about the child not performing as expected, or by the frus- 
trations of the child who perceives their parents as too controlling. For 
lower-achieving children, or parents with unrealistically high expectations, these 
tensions may have a negative impact on the child's self-esteem and performance at 
school. Helping with homework might also interfere with learning if parents are not 
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sufficiently equipped to help, if their instruction is very different from that of the 
school teacher, or if parents are overly involved, for example, completing assign- 
ments themselves (Cooper et al. 2000). 


2.3 Measuring Parental Involvement 


The literature review reveals that parental involvement is a multifaceted construct 
that includes a variety of parental behaviors and practices. It is important to view 
and measure parental involvement as a multidimensional concept, because research 
has indicated that some types of parental involvement may exert more influence on 
student performance (Jeynes 2005). Most studies tend to focus on parental 
involvement at home, at school or at both places (Powell et al. 2012). Bakker et al. 
(2007) discerned three types of parental involvement: (1) parents” behavior towards 
and activities with their child at home (home-based involvement), (2) parents' 
participation in school (school-based involvement), and (3) communication 
between parents and school (home-school communication). An additional distinc- 
tion sometimes used is school-initiated parental involvement versus parent-initiated 
involvement (Driessen et al. 2005). 

The most commonly used framework for parental involvement was developed 
by Epstein at the beginning of the 1990s (Epstein 1992; Manz et al. 2004). This 
framework refers mainly to school-initiated parental involvement, but is also 
characterized as a holistic approach to student learning: parents or families, schools 
and communities should work together as a partnership to create a positive learning 
environment for children (Mattingly et al. 2002). Epstein (1992) defined six types 
of parental involvement that can be influenced by the school; these also contain the 
three types of parental involvement identified by Bakker et al. (2007). Epstein 
(1992) stated that to increase the involvement of parents, schools and teachers 
should: 


(1) assist parents in child rearing skills (home-based involvement); 

(2) communicate with parents regularly (home-school communication); 

(3) involve parents in school volunteer opportunities (school-based involvement); 

(4) involve parents in home-based learning (home-based involvement); 

(5) involve parents in school-based decision making (home-school communica- 
tion); and 

(6) involve parents in  school-community collaborations (school-based 
involvement). 


The studies reviewed indicate that home-based involvement is mostly measured 
from the perspective of parents and sometimes from the perspective of students. 
With regard to communication between parents and schools, and school-based 
involvement, this can also be measured from the perspective of parents, and the 
perspective of school principals or teachers. Based on the literature review, we 
developed a general framework to categorize the studies reviewed (Table 2.1). 
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Table 2.1 General framework of parental involvement 


Parent Dimension 1 


perspective Home-based involvement: e.g., helping with homework, (early) literary 
activities, parent-child discussion of child's schooling, parenting style, 
parental monitoring and rule-setting, ensuring school readiness 


Dimension 2 


School-based involvement and home-school communication: e.g., attending 
parent meetings, participating in school activities, parent-teacher interaction 


Student Dimension 3 

perspective Home-based involvement: e.g., getting help with homework, parental 
monitoring and rule-setting, parent-child discussions about school 

School Dimension 4 

perspective School-based involvement and home-school communication: e.g., providing 
newsletters, asking parents to help at school, individual teacher-parents 
meetings 


In this framework, the different types of involvement and the perspectives by which 
they are perceived (i.e., parent, student, or school) are combined, resulting in four 
dimensions. We used this framework to categorize the meta-studies and empirical 
studies addressing the relationship between the involvement of parents and student 
attainment. 


2.4 The Relation Between Parental Involvement 
and Student Attainment 


As already mentioned, the goal of the literature review was not to present an 
exhaustive overview of all the available literature on this subject, but to gain insight 
into the dimensions of parental involvement that show the greatest potential to be 
related to student attainment, and to examine the role of cultural differences 
between countries. We identified large variation in outcomes and in the mechanisms 
used to conduct these studies (Tables 2.2, 2.3, 2.4, 2.5, 2.6 and 2.7). Most studies 
focused on more than one dimension; we recorded the main characteristics of the 
meta-studies (Tables 2.2 and 2.3) and individual studies (Tables 2.4, 2.5, 2.6 and 
2.T) for each dimension (whenever available). 

The outcomes of the meta-studies and the individual studies in this review 
indicate that parental involvement is generally positively correlated to or has pos- 
itive effects on student attainment. This is in agreement with the outcomes of a 
meta-synthesis of nine meta-analyses by Wilder (2013). With regard to the indi- 
vidual dimensions of parental involvement, the results are less definitive. For 
example, the association between parental involvement with homework and student 
achievement is positive in some studies, but non-existent or negative in others 
(Hoover-Dempsey et al. 2001; Patall et al. 2008). Based on a review of literature, 
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Table 2.2 Overview of meta-studies examining the relation between (dimensions of) parental 
involvement and student achievement, from the parent perspective 


Dimension 


Home-based 
involvement 


Authors 


Castro et al. 
(2015) 


Description 


Meta-analysis on 
parental 
involvement in 
kindergarten, 
primary and 
secondary 
education 


Dependent 
variable 


Achievement 
in various 
subjects 


Main conclusion 


In general positive 
effects: high 
academic 
aspirations, 
communication 
about school and 
stimulation of 
reading habits 
show strongest 
positive relation 
with student 
achievement. 
Supervision of 
homework is 
unrelated to 
achievement 


Erion (2006) 


Meta-analysis on 
the effects of 
parent tutoring in 
primary and 
secondary 
education 


Achievement 
in reading, 
language and 
math 


Parent tutoring is 
effective for 
improving 
achievement 


Fan and Chen 
(2001) 


Hill and Tyson 
(2009) 


Meta-analysis of 
experimental 
studies on the 
effects of parental 
involvement on 
achievement in 
primary and 
secondary 
education 


Meta-analysis on 
parental 
involvement in 


Achievement 
in various 
subjects 


Achievement 
in various 
subjects 


In general positive 
effects: parental 
academic 
aspirations show 
strongest positive 
relation with 
student 
achievement. 
Parental 
monitoring of 
homework shows 
a negative 
relation, direct aid 
with homework a 
positive relation. 
The correlation is 
lower if the 
achievement is 
measured in 
specific areas, 
such as math or 
reading 

In general positive 


effects, academic 
socialization 


(continued) 
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Table 2.2 (continued) 


Authors Description 


Dimension 


Dependent 
variable 


secondary 
education 


2 Literature Review 


Main conclusion 


showed the 
strongest positive 
relation with 
achievement. For 


homework 
involvement, the 
results are 
inconclusive 
Hoover-Dempsey | Meta-review, Achievement | Inconclusive, 
et al. (2001) parent in various positive and 
involvement in subjects negative relations 
homework in with or effects on 
primary and achievement 
secondary 
education 
Patall et al. (2008) | Meta-analysis, Achievement | In general small 
parent in various correlations, often 
involvement in subjects not significant and 


homework in 
kindergarten, 
primary education 
and secondary 
education 


they vary with 
students’ age. 
Setting rules 
about when and 
where homework 
should be done 
has the strongest 
positive relation 
with achievement 


Wilder (2013) 


Meta-synthesis of | Achievement 


In general positive 


nine in various effects. Parental 
meta-analyses on | subjects expectations 
parental (beliefs and 
involvement attitudes towards 
school teachers 
and subjects) 
showed the 
strongest 
relationship. No 
positive relation 
for help with 
homework 
School-based Castro et al. Meta-analysis on | Achievement | Involvement in 
involvement (2015) parental in various school activities is 
and involvement in subjects not related to 
home-school kindergarten, achievement 
communication primary and 
secondary 
education 


(continued) 
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Table 2.2 (continued) 


Dimension Main conclusion 


Authors Description Dependent 
variable 


Hill and Tyson Meta-analysis on | Achievement | In general positive 
(2009) parental in various effects 
involvement in subjects 
secondary 
education 
Wilder (2013) Meta-synthesis of | Achievement | In general positive 
nine in various effects 
meta-analyses on | subjects 
parental 
involvement 


McNeal Jr (2012) not only concluded that such inconsistencies have become a 
"standard" in the parental involvement literature, but also that these inconsistent 
findings cut across grade levels, measures of student attainment and time periods. 

One of the explanations for the mixed results is the complexity of the parental 
involvement concept (Fan and Chen 2001; Hill and Tyson 2009; Castro et al. 
2015). Both the meta-studies and our literature review indicated that measurement 
of parental involvement dimensions differed considerably among studies. 
According to Fan and Chen (2001), it is the multidimensional nature of parental 
involvement that has led to a lack of agreement about definitions and measurement 
inconsistencies, making it difficult to compare findings across studies. For example, 
"helping with homework" includes providing space and materials (such as a 


Table 2.3 Overview of meta-studies examining the relation between (dimensions of) parental 
involvement and student achievement, from the school perspective 


Dimension Authors Description Dependent Main 
variable conclusion 
School-based Mattingly Meta-review of Achievement Inconclusive 
involvement et al. evaluations of parental in various 
and (2002) involvement programs subjects 
home-school Sénéchal Meta-analysis of Achievement Positive 
communication — | and (quasi) experimental in reading effects found 
Young studies on literacy for parents 
(2008) encouragement and helping 
training programs for children to 
parent-child reading read 


activities and reading 
ability in kindergarten 
and primary school 
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Table 2.4 Overview of research literature examining the relation between home-based parental 
involvement (dimension 1) and student achievement from the parent perspective 


Authors Description Country Dependent Main conclusion 

variable 

Bakker et al. Survey, parents” Netherlands | Achievement in Parents’ 

(2007) and teachers’ different subjects perception is 
perception of positively 
parental related to 
involvement at reading 
home and at achievement 
school 

Barnard Survey, parents’ USA School dropout Parental 

(2004) and teachers’ and highest grade involvement at 
perception of completed at age home is not 
parental 20 related to 
involvement at educational 
home and at attainment 
school 

Driessen Survey, school Netherlands | Achievement in A negative 

et al. (2005) principals’ math and direct effect of 
perception of language, help with 
parental school-supportive homework on 
involvement at home climate, students’ 
school and well-being and achievement in 
policy towards self-confidence language and 
parental math 
involvement, 
parents’ 
perception of 
parental 
involvement at 
home 

Kloosterman Survey, parents’ Netherlands | Achievement in Parents reading 

et al. (2011) perception of math and activities are 
parental language most relevant for 
involvement at child’s language 
home and at achievement in 
school, teachers’ later grades of 
perception of primary school, 
parental parental reading 
involvement at instruction is 
school beneficial for 

language 
achievement at 
the start as well 
as during 
primary school 
Galindo and Survey, parents’ USA Gains in math and | Activities at 


Sheldon 
(2012) 


perception of 
parental 


reading in 
kindergarten 


home showed no 
association with 


(continued) 
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Authors Description Country Dependent Main conclusion 
variable 
involvement at math and 
home and at reading gains 
school 
Lau et al. Survey, parents’ China Achievement in Language and 
(2011) perception tests for readiness cognitive 
parental for school home-based 
involvement at including reading activities are 
home and at literacy at positively 
school kindergarten and related to 
entering primary reading literacy 
school level 
Myrberg and Survey, parents’ Sweden PIRLS-2001 Early reading 
Rosen (2009) | perception of reading literacy activities affect 
parental test, parents early reading 
involvement at perception of early | abilities. 
home (early reading abilities Educational 
reading level of the 
activities) parents has an 
effect on reading 
literacy, via 
books at home 
and early 
reading activities 
Powell et al. Survey, changes USA Achievement in The degree of 
(2012) in parents language, reading within-family 
involvement in literacy and math change in 
activities at several 
home and at dimensions of 
school from parents’ 
pre-K, home-based 
kindergarten and involvement is 
grade 1 related to math 
achievement but 
not to reading 
and literacy 
skills in grade 1 
Senechal and Survey, parents Canada Achievement in Storybook 
LeFevre perception of language and reading 
(2002) parental reading literacy at positively 
involvement at grade 1 and 3 related to 
home (early children’s 
literacy receptive 
activities) language 
development, 
teaching reading 
by parents 
positively 


(continued) 
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Table 2.4 (continued) 


Authors Description Country Dependent Main conclusion 
variable 


related to 
children (early) 
literacy skills, 


parental 
involvement not 
related to 
phonological 
awareness skills 
Stylianides Survey, USA Achievement in Low 
and parent-child different subjects parent-child 
Stylianides interactions in kindergarten interaction 
(2011) (including early relates 
literacy negatively with 
activities) achievement 
(incl. reading) 
Xu et al. Survey, parents USA Self-regulated TV-rules and 
(2010) perception learning perceived ` | help with 
parental by teachers, homework 
involvement at achievement in negative relation 
home and at reading literacy with reading 
school achievement. 


Involvement in 
school, parental 
educational 
expectations, 
engaging 
children in their 
homework and 
encouraging 
children’s’ 
extracurricular 
participation 
positive relation 
with reading 
achievement 


computer), monitoring whether homework is completed, having rules about when 
and where homework is done, and responding to questions and providing direct 
homework instruction (Hoover-Dempsey et al. 2001). In the meta-study of 
Hoover-Dempsey et al. (2001), it was clear that some of these dimensions were 
more effective than others; this may account for the mixed results. 

Not only is the measurement of dimensions of parental involvement often very 
different among studies, the explanation or justification for the measurements used 
is also frequently absent. Although many studies use Epstein’s (1992) framework as 
a starting point, most scholars in the field have developed their own indicators for 
parental involvement, and do not seem interested in developing or using existing 
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Table 2.5 Overview of research literature examining the relation between school-based 
involvement and home-school communication (dimension 2) and student achievement from the 


parent perspective 


Authors Description Country Dependent Main results 
variable 

Bakker et al. Survey, parents’ Netherlands | Achievement in Contact with the 

(2007) and teachers’ different subjects school is 
perception of negatively related 
parental to achievement in 
involvement at general, there is 
home and at no relation for 
school reading 

achievement 

Barnard Survey, parents’ USA School dropout Parent 

(2004) and teachers’ and highest involvement at 
perception of grade completed school is not 
parental at age 20 correlated to 
involvement at educational 
home and at attainment 
school 

Domina Survey, mothers’ USA Cognitive A positive effect 

(2005) perception of (reading and for preventing 
parental math) and behavioral 
involvement at behavioral problems was 
school and development found, but no 
students effect of 
perception of school-based 
parental involvement on 
involvement at achievement 
home 

Galindo and Survey, parents’ USA Gains in math Activities at 

Sheldon perception of and reading in school show 

(2012) parental kindergarten positive 
involvement at association with 
home and at math and reading 
school gains 

Kloosterman Survey, parents’ Netherlands | Achievement in School-based 

et al. (2011) perception of math and involvement 
parental language positively affects 


involvement at 
home and at 
school, teachers’ 
perception of 
parental 
involvement at 
school 


students’ 
language and 
math 
achievement at 
start of primary 
school, but the 
impact 
diminishes in 
subsequent 
primary school 
years 
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Table 2.5 (continued) 
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Authors Description Country Dependent Main results 
variable 
Lau et al. Survey, parents” China Achievement in Communication 
(2011) perception school readiness with the school is 
parental tests including positively related 
involvement at reading literacy to reading 
home and at at kindergarten literacy 
school and entering 
primary school 
level 
Okpala et al. Survey, parents’ USA Math Parental 
(2001) volunteering achievement volunteering 
hours at school grade 3-12 hours at school is 
not related to 
math 
achievement 
Powell et al. Survey, changes USA Achievement in The degree of 


(2012) 


in parents” 
involvement in 
activities at home 
and at school 
from pre-K, 
kindergarten and 
grade 1 


language reading 
literacy and math 


within-family 
change in several 
dimensions of 
parents” 
school-based 
involvement is 
related to math 
achievement but 
not to reading 
and literacy skills 
in grade 1 


empirically-tested measures and scales. This is an important limitation in con- 
ducting meta-studies, and makes it complicated to draw general conclusions about 
the relationship between dimensions of parental involvement and student 
attainment. 

However, the measurement of parental involvement is not the only reason why 
parental involvement is a complex concept. Although it seems logical to assume 
that parental support has positive effects on student achievement, it may also be 
reasonable to assume that low-achieving children need and receive more support 
and interference from their parents, resulting in a negative influence (Castro et al. 
2015). This phenomenon is called the “reactive hypothesis,” where parents of 
students with learning or behavioral problems react by intensifying their involve- 
ment in their children’s education (McNeal Jr 2012). McNeal Jr (2012) examined 
the relationship between achievement and parental involvement from grade 8 to 
grade 12 and concluded that there was no empirical evidence to support the reactive 
hypothesis, which seems to be largely championed by scholars who may be 
unwilling to believe that parental involvement could (also) have negative effects 
(McNeal Jr 2012). Conversely, some studies do seem to provide support for the 
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Table 2.6 Overview of the research literature on the relation between home-based parental 
involvement (dimension 3) and student achievement, student perspective 


Authors Description Country Dependent Main results 
variable 
Domina Survey, mothers’ Cognitive No effect found for 
(2005) perception of (reading and home-based 
parental math) and involvement on 
involvement at behavioral achievement 
primary school and development 
students perception 
of parental 
involvement at 
home 
Dumont Survey, students’ Achievement in Positive effects 
et al. perception of math and reading, found for perceived 
(2012) parental academic parental competence 
involvement with self-concept and to help with 
homework homework homework and 
self-efficacy parental support on 
student outcomes, 
negative effect of 
perceived 
homework conflict 
on achievement 


reactive hypothesis. When the association between parental involvement and 
achievement is controlled for the initial ability, intelligence or the SES of the 
student, the association is more likely to be positive (Wilder 2013; Castro et al. 
2015). 

A study by Domina (2005) found that the initial positive effects of parents’ 
involvement in school and at home on the reading and math achievement of pri- 
mary school students, changed to non-significant or even negative effects when 
controlled for student background characteristics. This indicates that the effects of 
student background characteristics on the relation between parental involvement 
and student outcomes are also unclear. While most studies show a positive rela- 
tionship between SES and parental involvement (Cooper et al. 2000; Fan and Chen 
2001; Mattingly et al. 2002), a study of Dutch parents of primary school students 
(Stoep et al. 2002) revealed that home-based involvement was higher among lower 
SES parents than among high SES parents. Yet another Dutch study found no 
relation between the educational level of the parents of primary school children and 
involvement in their child’s education (Bakker et al. 2007). 

Whether the reactive hypothesis should be rejected or not, the lack of consensus 
confirms the difficulties introduced in proposing that parental involvement influ- 
ences student attainment. One of the meta-studies focused specifically on the effects 
of parental involvement programs offered by schools (Mattingly et al. 2002). The 
results were disappointing; according to Mattingly et al. (2002), studies evaluating 
the effectiveness of these programs did not provide convincing evidence that these 
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Table 2.7 Overview of the research literature on the relation between school-based involvement 
and home-school communication (dimension 4) and student achievement, school perspective 


Authors Description Country Dependent Main results 
variable 
Bakker Survey, parents” Netherlands | Achievement in Teacher 
et al. and teachers’ different subjects perception of 
(2007) perception of parental 
parental involvement is 
involvement at positively related 
home and at school to reading 
achievement 
Barnard Survey, parents’ USA School dropout Teacher 
(2004) and teachers’ and highest grade perception of 
perception of completed at age parental 
parental 20 involvement is 
involvement at positively related 
home and at school to educational 
attainment 
Driessen Survey, primary Netherlands | Achievement in No direct effect on 
et al. school principals’ language and student outcomes 
(2005) perception of math, found for school 
parental school-supportive activities to 
involvement at home climate, involve parents 
school and policy well-being and 
towards parental self-confidence 
involvement, 
parents’ perception 
of parental 
involvement at 
home 
programs had any positive effects on student achievement or other 


achievement-related outcomes. Furthermore, most of these studies were correla- 
tional or had other design limitations, which made it difficult to determine whether 
there was a causal effect. 

Another limitation that may have contributed to the inconsistencies in outcomes 
apparent in this field of research was mentioned by Hoover-Dempsey et al. (2005). 
In their review of parental motivations for involvement, they recognized that much 
research on parental involvement has relied on single-source reports. Parents’ 
perception of their school-based involvement can be different from the school’s 
perception of parental involvement. Students and parents may also differ in their 
perception of home-based involvement. Using parent, as well as student and school 
information, enables triangulation of essential perspectives on involvement, and 
thereby allows a more precise determination of parental involvement and its 
influence on student outcomes. 

In our literature review, we also focused on whether cultural differences in 
parental involvement could also explain some of the differing research results. 
However, most of the studies reviewed were conducted in the USA, and none of the 
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meta-studies explicitly incorporated this factor. A study comparing the definition of 
parental involvement for  European-American parents versus that of 
immigrant-Chinese parents in the USA revealed that European-American parents 
were more involved in school-based activities while the immigrant-Chinese parents 
focused more on systematic teaching of their children at home (Huntsinger and Jose 
2009). Another study compared the home-based and school-based parental 
involvement of Jewish and Arab parents in Israel, using the parental involvement 
model of Hoover-Dempsey and Sandler (1995) for both populations. This study 
showed that Arab parents were more involved with their child’s education than 
Jewish parents. However, the intensity of the invitation of the child to be involved 
(“your child has asked you to ...”) was higher among Jewish parents. Although 
these two examples did not compare different countries, they do suggest that is very 
likely that cultural differences in the perception of parental involvement exist. For 
international comparative studies in education, such as PIRLS, the Programme for 
International Student Assessment (PISA), or the Trends in International 
Mathematics and Science Study (TIMSS), this could mean that different parental 
perceptions of what is important for the education of their child can also have 
consequences for how survey questions about parental involvement are interpreted. 

The next chapters report the results of two strains of analysis exploring the 
possible cultural differences in perceptions of parental involvement among the 
PIRLS countries: first by looking at the country-item interactions in scales mea- 
suring components of parental involvement, then by the variation in the relation 
between different parental involvement components and student achievement across 
countries. 
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Chapter 3 
Parental Involvement in PIRLS-2011 


Our analysis of the PIRLS-2011 data of 41 countries was guided by the analytic 
framework (Table 3.1), which was based on the general framework (Table 2.1) we 
used to categorize the studies included in the literature review. The analytic 
framework matches the available indicators of parental involvement in the 
PIRLS-2011 data (Table 3.2). 

The first dimension, i.e. home-based involvement from the perspective of par- 
ents, was split into two components or indicators: early literacy activities and help 
with homework. The early literacy activities component is especially well measured 
by the PIRLS home questionnaire. In the international reports of PIRLS-2011, early 
literacy activities is the only component reported as a scale, with Cronbach's alphas 
ranging from 0.70 (Czech Republic, Hungary, Italy and Oman) to 0.88 (Romania), 
indicating high reliability (Martin and Mullis 2012). Although the international 
report does not report the scale statistics on the items regarding parental help with 
homework, this component is well addressed, as a total of eight items ask about 
these practices. 

To consider the dimension of school-based involvement and home-school 
communication from the parent's perspective (component 3 in the analytic 
framework), three relevant items were selected from the home questionnaire. The 
number of items for this indicator is low, but the items do seem highly relevant to 
this context. The student's perception of parental involvement and the school's 
practices on parental involvement (corresponding to components 4 and 5 in the 
analytic framework) are measured by five items in the student questionnaire and 15 
items in the school questionnaire, respectively. Finally, we established the outcome 
variable and control variables for the structural multilevel model tested later 
(Table 3.2): the PIRLS reading literacy achievement scores, gender of the student, 
and, as approximates for SES, both books at home and highest level of education of 
the parents. 
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Table 3.1 Analytic framework for parental involvement for secondary analyses of PIRLS-2011 


Perspective | Dimension Components 
Parent Dimension 1 Component 1 Home Question 2: 
Home-based Early literacy questionnaire | A-I 
involvement activities before 
beginning primary 
school 
Dimension 1 Component 2 Home Question 9 
Home-based Help with questionnaire 
involvement homework 
Dimension 2 Component 3 Home Question 10: 
School-based School practices questionnaire |A, B, E 
involvement and on parental 
home-school involvement 
communication 
Student Dimension 3 Component 4 Student Question G7* 
Home-based Student perception | Questionnaire Question 
involvement of parental R9:C 
involvement 
School Dimension 4 Component 5 School Question 11 
School-based School practices questionnaire | Question 12: 
involvement and on parental E-F 
home-school involvement 
communication 


Note The PIRLS 2011 questionnaires can be retrieved from http://timssandpirls.bc.edu/pirls2011/ 
international-contextual-q.html (IEA & TIMSS and PIRLS International Study Center 2011) 
"The PIRLS 2011 student questionnaire consists of multiple sections. Item G7 refers to item 7 in 
the general part of the questionnaire, R9 to item 9 in the reading section 


The benefit of using data from a large-scale assessment study such as PIRLS is 
not only the richness in data resulting from achievement tests, as well as student, 
home, teacher and school questionnaires, but also, obviously, the large number of 
countries for which these data are available. For this study, we initially considered 
the data from 43 countries participating in PIRLS-2011; countries not meeting the 
required response rate or for which the average achievement was not reliably 
measured were excluded. However, two countries, England and the USA, did not 
administer the home questionnaire and were therefore not included in the scale 
analyses for components 1—3, using items from the home questionnaire. 
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Table 3.2 Overview of components and items for secondary analyses of PIRLS-2011 


Component 


1 Early literacy 
activities before 
beginning primary 
school (home-based 
involvement) 


Question 


Before your child 
began primary school, 
how often did you or 
someone else in your 
home do the following 
activities with him or 
her? 


Item number 
per component 


Item in 
international 
datasets 


Number 
of 
response 
categories 


Read books 


ASBHO2A 


3a 


Tell stories 


ASBHO2B 


38 


Sing songs 


ASBHO2C 


3? 


Play with alphabet toys 


ASBHO2D 


3? 


Talk about things you 
had done 
Talk about what you 
had read 


ASBHO2E 


ASBHO2F 


3a 


Play word games 


Write letters or words 


ASBHO2G 
ASBHO2H 


3? 


Read aloud signs and 
labels 


ASBHO?2I 


ER 


(Source: Home 
questionnaire) 


2 Help with homework 
(home-based 
involvement) 


How often do you or 
someone else in your 
home do the following 
things with your child? 


Discuss my child’s 
schoolwork with 


him/her 


ASBHO9A 


Help my child with 
his/her schoolwork 


ASBHO9B 


Make sure my child 
sets aside time to do 
his/her homework 


ASBHO9C 


Ask my child what 
he/she learned in 
school 


ASBHO9D 


Check if my child has 
done his/her 
homework 


ASBHO9E 


Ab 


Help my child practice 


his/her reading 


ASBH09F 


Ab 
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Table 3.2 (continued) 


Component 


Question 


Help my child practice 
his/her math skills 
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Item number 
per component 


Item in 
international 
datasets 


ASBH09G 


Number 
of 
response 
categories 


Talk with my child 
about what he/she is 
reading 


ASBH09H 


(Source: Home 
questionnaire) 


3 School practices on 
parental involvement, 
parent perspective 
(home-school 
communication) 


What do you think of 
your child's school? 


My child's school 
includes me in my 
child's education 


ASBHIOA 


4° 


My child’s school 
should make a greater 
effort to include me in 
my child’s education 


ASBH10B 


4° 


My child’s school 
should do better at 
keeping me informed 
of his/her progress 


ASBHIOE 


4° 


(Source: Home 
questionnaire) 


4 Parental 
involvement, student 
perspective (home- 
based involvement) 


How often do the 
Jollowing things 
happen at home? 


My parents ask me 
what I am learning in 
school 


ASBG07A 


I talk about my 
schoolwork with my 
parents 

My parents make sure 
that I set aside time for 
my homework 


ASBGO7B 


ASBGO07C 


Ab 


My parents check if I 
do my homework 


ASBG07D 


Ab 


Do you read for any of 
the following reasons? 


My parents like it 
when I read 


ASBR09C 


4° 


(Source: Student 
questionnaire) 


(continued) 
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Table 3.2 (continued) 


Component 


5 School practices on 
parental involvement, 
school perspective 
(home-school 
communication, 
school-based 
involvement) 


Question 


How often does your 
school do the following 
for parents concerning 
individual students? 


Item in 
international 
datasets 


Item number 
per component 


29 


Number 
of 
response 
categories 


Inform parents about 
their child’s learning 
progress 


Inform parents about 
the behavior and 
well-being of their 
child at school 


1 ACBGIIAA 


2 ACBGIIAB 


4d 


Ad 


Discuss parents” 
concerns or wishes 
about their child’s 
learning 


3 ACBG11AC 


Support individual 
parents in helping their 
child with schoolwork 


4 ACBGIIAD 


How often does your 
school ask parents to 
do the following? 


Volunteer for school 
projects, programs, and 
trips 


5 ACBGIIBA 


4d 


Serve on school 
committees 


6 ACBGIIBB 


Ad 


How often does your 
school do the following 
Jor parents in general? 


Inform parents about 
the overall academic 
achievement of the 
school 


7 ACBGIICA 


Inform parents about 
school 
accomplishments 


8 ACBGIICB 


Ad 
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Table 3.2 (continued) 


Component 


Question 


Inform parents about 
the educational goals 
and pedagogic 
principles of the school 
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Item number 
per component 


Item in 
international 
datasets 


ACBG11CC 


Number 
of 
response 
categories 


Inform parents about 
the rules of the school 


10 


ACBG11CD 


Discuss parents” 
concerns or wishes 
about the school’s 
organization 


11 


ACBGIICE 


Provide parents with 
additional learning 
materials 


12 


ACBGIICF 


Organize workshops or 
seminars for parents on 
learning or pedagogical 
issues 


13 


ACBG11CG 


How would you 
characterize each of 
the following within 
your school? 


Parental support for 
student achievement 


14 


ACBG12E 


5° 


Parental involvement 
in school activities 


15 


ACBG12F 


5° 


(Source: School 
questionnaire) 


Socioeconomic status 
(books at home, 
highest level of 
education completed 
by parent) 


About how many 
books are there in your 
home? 


ASBH14 


What is the highest 
level of education 
completed by the 
child’s father? 


ASBHI7A 


What is the highest 
level of education 
completed by the 
child’s mother? 


ASBH17B 


98 


(Source: Home 
questionnaire) 
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Table 3.2 (continued) 


Component Question Item number Item in Number 
per component | international | of 
datasets response 
categories 

Gender (Source: Student ASBG01 2 
questionnaire) 

Reading literacy (Source: PIRLS reading comprehension | R11F01M to R31G14M 
assessment) 


Note The datasets are described in detail in Foy and Drucker (2013) 

“Category labels are: 0 = Often, 1 = Sometimes, 2 = Never or almost never 

PCategory labels are: 0 = Every day or almost every day, 1 = Once or twice a week, 2 = Once or 
twice a month, 3 = Never or almost never 

“Category labels are: 0 = Agree a lot, 1 = Agree a little, 2 = Disagree a little, 3 = Disagree a lot 
“Category labels are, after recoding: 0 = More than three times a year, 1 = Two to three times a 
year, 2 = Once a year, 3 = Never 

“Category labels are: 0 = Very high, 1 = High, 2 = Medium, 3 = Low, 4 = Very low 

‘Category labels are: 0 = 1-10, 1 = 11-25, 2 = 26-100, 3 = 101—200, 4 = More than 200 
Category labels are: 0 = Did not go to school, 1 = Some ISCED level 1 or 2, 2 = ISCED level 2, 
3 = ISCED level 3, 4 = ISCED level 4, 5 = ISCED level 5B, 6 = ISCED level 5A, 7 = Beyond 
ISCED level 5A, 8 = Not applicable 
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Chapter 4 
Modeling Parental Involvement 


In this chapter, we first outline the models used and the estimation and testing 
procedures employed, and then summarize the results revealed by these models. 


4.1 Estimation and Testing Procedures 


The procedures we used for parameter estimation and evaluation of model fit are 
based on marginal maximum likelihood (MML). Most of the procedures we discuss 
are documented in more detail elsewhere (see Bock and Aitkin 1981; Bock et al. 
1988; Gibbons and Hedeker 1992; Glas 1999; Adams and Wu 2006; De Jong et al. 
2007; Jennrich and Bentler 2011; Glas and Jehangir 2014). We used the public 
domain software package MIRT (Glas 2010) in the calculations. Additional esti- 
mation and testing procedures were used for the bi-factor model, with unidimen- 
sional models as special cases, and random item parameters as a generalization. 


4.1.1 MML Estimation 


The bi-factor model used in this study was in two parts: a measurement model (i.e., 
an IRT model) and a structural model. The measurement model pertains to a 
polytomously-scored response of a student n to an item 7. The possible item scores 
range from 0 to m; and the score of student n on item i is denoted by the variables 
Xn (j = L, ..., mj) where x,,; = 1 if the response is in category 1 and zero otherwise. 
Note that m; has an index i, which indicates that the maximum score of items can 
differ. 

We describe the procedure for the bi-factor model, combined with the partial 
credit model (PCM; Masters 1982) and generalized partial credit model (GPCM; 
Muraki 1992) as IRT models, since these two models were the ones we selected for 
the present study. However, the theory also applies to other IRT models, such as the 
unidimensional PCM and GPCM, the graded response model (Samejima 1969), the 
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sequential model (Tutz 1990), and other versions of these models with random item 
parameters instead of fixed item parameters. 

In the bi-factor GPCM, the probability of scoring in category j (j = 0, ..., mj) is 
given by 


j 
exp Es EIN + Gig(n) Ü, (m) T 2 


h=1 


GG = psg = UO, a, b) €: SS i 
1+ > exp E i0 Ono + Gig(n) Deia) = ba) 
k=1 h=1 


(4.1) 


where, @,o is the score of a student n on the latent scale pertaining to all countries, 
Ongin) is the score on a country specific latent dimension, and the index g(n) indi- 
cates the country to which student n belongs. Further, ajo and aj, are the factor 
loadings of item i on these two dimensions, and bj, (h = 1, ..., mj) is the item 
location parameter. The location parameter bj, is the position on the latent scale, 
where it is assumed that summations such as h = 1 to 0 result in zero. The uni- 
dimensional GPCM lacks the country-specific dimensions 0,,(, and the associated 
factor loadings dor. Further, the PCM is obtained by fixing all item parameters ajo 
to one. 

The formula for the response probability and subsequent derivations can be 
simplified by introducing the re-parametrization d;; = Xj, — by, and by defining dis, 
as the inner product of the vectors (ajo, Oste) and (Ono, Bastel, respectively. Thus, 
Eq. (4.1) becomes 


exp (ia!,0n — di) 
pij(0,) m m; 
1+ > exp (ro, — dx) 

k=1 


(4.2) 


The 9-dimension is the general dimension that pertains to all countries and is the 
basis for the comparison of the countries. The 6,-dimensions are the 
country-specific dimensions, and the factor loadings on these dimensions give an 
indication of country-by-item interaction. It is assumed that within each country, the 
dimensions On and 0, have a bi-variate normal distribution N (0,0, Ong; Hg» Z,). For 
the two-dimensional country mean 4, = (Hgo, Hg), it holds that the mean on the 
second dimension is fixed at zero, that is u, = 0. The covariance matrix is given by 


c? 0 
x=] ] 


In the unidimensional GPCM and PCM, the latent student parameters Ou have a 
univariate normal distribution with a mean 4, and a variance a. Finally, random 
item parameters are obtained by introducing independent multivariate normal 
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distributions on the parameters for each item (for further details, please consult De 
Jong et al. 2007). 

The present application of the bi-factor model is not standard, but an extension 
of the basic model. Thus, the technical details on the estimation equations, 
expressions for the covariance matrix of the estimates, and tests of model fit, are 
also provided (see Appendix A). 


4.1.2 Detection and Modeling of Differential Item 
Functioning 


Part of the process of establishing the construct validity of a scale may consist of 
showing that the scale fits an IRT model. In the present study, the focus is on 
country-specific CDIF. CDIF can be detected using Lagrange multiplier (LM) test 
statistics (Rao 1947; see also, Aitchison and Silvey 1958) and CDIF can be 
modeled using country-specific item parameters. Glas and Jehangir (2014) already 
showed the feasibility of the method using PISA data, although in the slightly 
simpler framework of one-dimensional IRT models. The method is implemented in 
the public domain software package MIRT (Glas 2010). LM tests have been pre- 
viously applied to IRT frameworks (Glas 1999; Glas and Falcón 2003; Glas and 
Dagohoy 2007). Our primary interest is not in the actual outcome of the LM test, 
because due to the very large sample sizes in educational surveys even the smallest 
model violation, that is, the smallest amount of differential item functioning (DIF), 
will be significant. The reason for adopting the framework of the LM test is that it 
clarifies the connection between the model violations, and observations and 
expectations used to detect DIF. Further, because it produces comprehensible and 
well-founded expressions for model expectations, the value of the LM test statistic 
can be used as measure of the effect size of DIF, and the procedure can be easily 
generalized to a broad class of IRT models. 

To define the test and the associated residuals, we define a background variable 

_ J 1 if person n belongs to country c, 
ae = e if person n does not belong to country c. 

The LM test targets the null-hypothesis of no DIF, namely the null-hypothesis 
where ó; = 0. The LM test statistic is computed using the MML estimates of the 
null-model, where ó; is not estimated. The test is based on evaluation of the 
first-order derivatives of the marginal likelihood with respect to 6; evaluated at 
6; = 0 (see Glas 1999). If the first-order derivative in this point is large, the MML 
estimate of 6; is far removed from zero, and the test is significant. If the first-order 
derivative in this point is small, the MML estimate of 6; is probably close to zero 
and the test is not significant. The actual LM statistic is the squared first-order 
derivative divided by its estimated variance, and it has an asymptotic chi-squared 
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distribution with one degree of freedom. However, as already discussed, the pri- 
mary interest is not so much in the test itself, but in the information it provides 
regarding the fit between the data and the model. 

For a general definition of the approach, which also pertains to 
polytomously-scored items, the covariates y,¿ (c = 1, ..., C) should be defined. 
Special cases leading to specific DIF statistics are given later. The covariates may 
be separately observed person characteristics, but they may also depend on the 
observed response pattern, but without the response to the item ; targeted. 

The LM approach can be outlined using the bi-factor GPCM; the special cases 
for the unidimensional PCM and GPCM are obtained if the restrictions denoted 
above are invoked. The probability of a response is given by a generalization of the 
bi-factor GPCM, namely, 


exp (it, m dij +j) EN 
pi(0,) = m; ° 
1+ > exp e = di +k 25 EN 


k=1 


For one so-called reference country, the covariate y,, is equal to zero. This 
country serves as a baseline where the bi-factor GPCM with item parameters a and 
b holds. In the other C-1 countries, the covariates y,, are equal to one. It can be 
shown (see Glas 1999) that the test statistic is based on the residuals 


N mi N mi 
> > Mac > > yn jE (P; (0,) Lo 2) 
n=1 j= n=1 j= 
= X (4.3) 
Kä Ync Kë Ync 
n=1 n=1 
for c = 1, ..., C-1. Dividing this residual by the number of respondents ~,y,,. 
produces residuals that are the differences between the observed and expected 
average item-total score in country c = 1, ..., C-1. The residual gauges so-called 


uniform DIF, in other words, the residual indicates whether the item total function 
(ITF) 2; jP;(0) is shifted for the item, namely whether there is item-by-country 
interaction. 

The LM statistic for the null-hypothesis ó; = 0 (c = 1, ..., C-1) is a quadratic 
form in the (C-1)-dimensional vector of residuals and the inverse of their covariance 
matrix (for details, see Glas 1999). It has an asymptotic chi-squared distribution 
with C-1 degrees of freedom. 

A special case of this procedure is obtained if one country serves as the focal 
country and all other countries serve as reference. Then the model under the 
alternative hypothesis has only one additional parameter, 6;, and the associated LM 
statistic has an asymptotic chi-squared distribution with one degree of freedom. 

Items that show the worst misfit, based on their value of the LM statistic and 
residuals, are given country-specific item parameters. From a practical point of 


4.1 Estimation and Testing Procedures 37 


view, defining country-specific item parameters is equivalent to defining an 
incomplete design where the DIF item is split into a number of virtual items, and 
where each virtual item is considered as administered in a specific country. The 
resulting design can be analyzed using IRT software that supports the analysis of 
data collected in an incomplete design. We here refer to items with country-specific 
parameters as split items. 

The method is motivated by the assumption that a substantial part of the items 
function the same in all countries and a limited number of items have CDIF. In the 
IRT model, it is assumed that all items pertain to the same latent variable d Items 
without CDIF have the same item parameters in every country. However, items 
with CDIF have item parameters that differ across countries. These items refer to 
the same latent variable d as all the other items, but their location on the scale differs 
across countries. For instance, the number of cars in the family may be a good 
indicator of wealth, but the actual number of cars at a certain level of wealth may 
vary across countries, or even within countries. Having a car in the inner city of 
Amsterdam is clearly a sign of wealth, but, in the rural eastern part of the 
Netherlands, an equivalent level of wealth would probably result in the ownership 
of three cars. 

The number of items given country-specific item parameters is a matter of choice 
where two considerations are relevant. First, there should remain a sufficient 
number of anchor items in the scale. Second, the model including the split items 
should fit the data. DIF statistics no longer apply to the split items. However, the fit 
of the item response curve of an individual item, say item i, can be evaluated using 
the test for non-uniform DIF described earlier, but using a model including 
country-specific items parameters. So, in this application too, test-score ranges are 
used as proxies for locations on the @ scale, and the test evaluates whether the 
model with the country-specific item parameters can properly predict the ITF. 


4.20 Results of Modeling Country-Specific Differential Item 
Functioning 


We here provide descriptive statistics at country level for each of the five parental 
involvement components under the PCM and GPCM, including sample size and 
estimated global reliability (Tables 4.1, 4.2, 4.3, 4.4 and 4.5). Sample sizes for the 
first four components (early literacy activities, help with homework, school prac- 
tices on parental involvement from a parental perspective, and parental involvement 
from a student perspective) were taken from the PIRLS home and student data, 
providing a significantly larger sample than that available for the last component 
(school practices on parental involvement, school perspective), where data were 
derived from the PIRLS school questionnaire. The GPCM rarely improved global 
reliability. Components 1 (early literacy activities), 2 (help with homework), and 5 
(school practices on parental involvement, school perspective) were evaluated using 
nine, eight, and 15 items, respectively (see also Table 3.2). Their global reliability is 
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Table 4.1 Country characteristics component 1: early literacy activities before beginning primary 

school 

Country N x PCM GPCM 

(0 e |p luo loo |» 
0.36 098 0.74 

—049 1.49 0.77 
0.08 094 0.74 
029 094 0.74 
0.12 146 0.85 

—044 |1.14 0.76 
0.90 |1.03 0.78 


Azerbaijan, Republic of 


Australia 


Austria 


Belgium (French) 


Bulgaria 


Canada 


Chinese Taipei 


Colombia 0.13 |1.10 0.77 
Croatia —0.35 0.90 | 0.69 
Czech Republic —0.09 [0.84 | 0.69 
Denmark 0.18 (0.90 0.73 
Finland 024 |0.74 0.65 
France 0.11 |0.95 | 0.74 
Georgia —044 | 1.02 | 0.72 
Germany —0.02 | 0.89 | 0.71 
Hong Kong, SAR 0.91 | 0.90 | 0.74 
Hungary —0.12 | 0.85 | 0.69 
Indonesia 0.45 | 0.94 | 0.75 


Iran, Islamic Republic of 075 | 0.99 | 0.76 


Ireland —043 | 1.14 | 0.76 
Israel —0.30 | 1.03 | 0.74 
Italy —0.20 |0.93 | 0.71 
Lithuania 0.01 |0.90 | 0.71 
Malta —0.17 | 1.06 | 0.76 
Netherlands —0.02 |0.89 | 0.71 
New Zealand —0.54 | 1.22 | 0.77 
Norway 0.08 |0.90 | 0.72 
Northern Ireland —0.68 | 1.18 | 0.75 
Poland -0.20 |0.92 |0.71 
Portugal 0.04 |1.01 |0.76 
Qatar 0.30 | 1.00 | 0.76 
Romania —0.12 | 1.46 | 0.84 


—0.68 | 1.09 | 0.73 
0.36 |0.97 | 0.75 


Russian Federation 
Saudi Arabia 


Singapore 047 | 1.15 | 0.81 
Slovak Republic —0.23 | 1.00 | 0.74 
Slovenia —0.31 |0.94 | 0.71 
Spain —0.16 |0.95 | 0.73 
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Table 4.1 (continued) 


Sweden 


39 


Trinidad and Tobago 


—0.29 


1.08 


United Arab Emirates 


0.32 


0.96 


Note N is the sample size, and X the observed mean score on the component. (0) is the estimated 
mean, o(0) is the standard deviation, and p is the estimated global reliability under the partial credit 
model (PCM) or the generalized partial credit model (GPCM) 


Table 4.2 Country characteristics component 2: help with homework 


Country N X PCM GPCM 
wm "em Je uf "em |p 

Azerbaijan, Republic of —0.63 |1.30 | 0.76 
Australia 0.33 | 0.80 | 0.80 
Austria 0.57 | 0.81 | 0.82 
Belgium (French) —0.30 | 1.16 | 0.79 
Bulgaria —0.13 | 1.50 | 0.84 
Canada —0.02 |0.92 | 0.78 
Chinese Taipei 033 | 1.00 | 0.84 
Colombia —0.46 |1.12 | 0.76 
Croatia 0.32 | 0.88 | 0.82 
Czech Republic 022 |0.74 | 0.76 
Denmark 0.36 | 0.82 | 0.80 
Finland 0.96 |0.64 | 0.80 
France —0.15 | 0.99 | 0.78 
Georgia —0.53 |123 | 0.77 
Germany 0.49 090 | 0.84 
Hong Kong, SAR 0.28 | 1.13 | 0.85 
Hungary —0.15 |1.13 | 0.80 
Indonesia —0.21 | 1.10 | 0.79 
Iran, Islamic Republic of 0.07 | 1.01 | 0.81 
Ireland —0.46 | 1.11 | 0.76 
Israel 043 | 0.92 | 0.83 
Italy —0.12 | 1.05 | 0.80 
Lithuania 0.35 | 0.92 | 0.83 
Malta 0.04 093 | 0.79 
Netherlands 1.00 076 | 0.85 
New Zealand 0.26 |0.94 | 0.82 
Norway —0.54 | 0.93 | 0.70 
Northern Ireland 2908 | 3.56 |-020 | 1.39 |0.75 |-0.15 |0.91 | 0.76 


(continued) 
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Poland 

Portugal 0.79 | -0.23 | 1.14 | 0.80 
Qatar 0.76 |-0.35 | 1.04 | 0.76 
Romania 0.80 |-0.50 | 1.52 | 0.81 
Russian Federation 0.76 |-028 | 1.01 | 0.77 
Saudi Arabia 079 |-023 |1.12 | 0.79 
Singapore 0.83 033 | 0.99 | 0.84 
Slovak Republic 0.81 022 099 | 0.83 
Slovenia 0.80 0.18 |0.96 | 0.82 
Spain 0.76 |-0.43 | 1.17 | 0.77 
Sweden 0.79 0.19 |0.88 | 0.80 
Trinidad and Tobago 072 |-0.68 | 1.12 | 0.73 
United Arab Emirates 0.76 |-0.40 | 1.07 | 0.76 


Note N is the sample size, and X the observed mean score on the component. ut) is the estimated 
mean, o(0) is the standard deviation, and p is the estimated global reliability under the partial credit 
model (PCM) or the generalized partial credit model (GPCM) 


Table 4.3 Country characteristics component 3: school practices on parental involvement, parent 


perspective 
Country N X PCM GPCM 

OMEO vg CONT 
Azerbaijan, Republic of 4401 |0.79 —202 | 1.40 
Australia 3185 |3.51 0.39 0.13 
Austria 4349 |4.11 0.63 |0.19 
Belgium (French) 3269 |4.14 0.63 |0.13 
Bulgaria 5029 |2.70 |—0.01 |0.58 —0.07 | 4.62 |0.67 
Canada 18567 |3.66 0.45 |0.16 3.58 1.29 |0.43 
Chinese Taipei 4189 |1.85 |—-0.35 |023 -2.TI 1.84 |0.54 
Colombia 3738 131 |-131 |131 —10.37 |10.33 |0.62 
Croatia 4478 | 3.05 0.18 | 0.46 1.43 3.65 | 0.65 
Czech Republic 4316 | 3.68 0.46 | 0.15 3.65 1.16 |0.40 
Denmark 4243 4.03 0.58 |0.13 4.60 101 | 0.37 
Finland 4348 | 4.53 0.73 | 0.10 5.77 | 0.79 | 0.28 
France 3961 | 3.86 0.52 | 0.10 4.14 | 0.81 |0.27 
Georgia 4483 | 1.63 | —0.80 | 0.96 —6.31 7.58 |0.64 
Germany 3097 | 3.88 0.54 | 0.13 4.25 | 0.99 | 0.36 
Hong Kong, SAR 3593 |1.51 |—0.60 |0.33 —4.15 2.64 | 0.59 
Hungary 4793 | 3.38 0.36 | 0.22 2.83 1.77 | 0.52 


(continued) 
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Table 4.3 (continued) 


Indonesia 4549 |0.86 |-1.65 | 1.09 

Iran, Islamic Republic of | 5608 | 1.34 |—1.00 | 0.89 

Ireland 4187 | 3.44 0.37 | 0.36 

Israel 3188 | 2.47 |-0.11 | 0.55 

Italy 3755 | 3.61 043 | 0.11 

Lithuania 4347 | 3.45 0.38 | 0.22 

Malta 3188 | 2.35 |-025 | 0.80 

Netherlands 2265 | 4.39 0.70 | 0.13 

New Zealand 3362 | 3.56 0.42 | 0.23 

Norway 2091 | 3.92 0.55 | 0.19 

Northern Ireland 2884 | 3.49 0.38 | 0.11 

Poland 4790 | 3.25 0.32 | 0.15 

Portugal 3745 | 3.60 043 | 0.11 

Qatar 3610 | 1.87 |-0.39 | 0.40 

Romania 4477 | 2.00 |-0.52 | 0.90 

Russian Federation 4331 | 3.25 0.31 | 0.19 

Saudi Arabia 4306 | 1.39 |-1.05 | 1.03 

Singapore 6145 | 2.03 |-023 | 0.18 

Slovak Republic 5344 | 3.08 023 | 0.23 

Slovenia 

Spain 

Sweden 3974 | 3.59 043 | 0.13 0.35 
Trinidad and Tobago 3328 |2.03 |-0.51 |0.96 0.67 
United Arab Emirates 13061 168 |—0.59 | 0.62 0.64 


Note N is the sample size, and X the observed mean score on the component. ur) is the estimated 
mean, o(0) is the standard deviation, and p is the estimated global reliability under the partial credit 
model (PCM) or the generalized partial credit model (GPCM) 


generally >0.70, which is an acceptable level for country inferences. A value of 
0.80 is generally considered an acceptable reliability level for individual inferences, 
and for many combinations of components and countries, this level was attained. 
Components 3 (school practices on parental involvement, parental perspective) and 
4 (parental involvement from a student perspective), were evaluated using three 
items and five items, respectively; the global reliability of these estimates was thus 
correspondingly lower. 

We also investigated the item characteristics for each component (Tables 4.6, 4.7, 
4.8, 4.9 and 4.10). Local reliability, namely the extent to which different 6-values can 
be distinguished, was assessed using the “slope” parameter. The relatively high 
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Table 4.4 Country characteristics component 4: student perception of parental involvement 
PCM GPCM 
DEG Wf [e(0) [p 
—0.54 |1.18 0.51 
0.55 |0.81 | 0.58 
—0.28 |1.09 0.55 
—0.17 |1.00 | 0.57 
—0.29 |1.44 0.64 
0.13 |0.96 0.57 
0.79 |1.04 | 0.70 


Country 


Azerbaijan, Republic of 


Australia 


Austria 


Belgium (French) 


Bulgaria 


Canada 


Chinese Taipei 


Colombia —0.88 |1.40 0.53 
Croatia —0.06 0.88 | 0.50 
Czech Republic —0.62 |1.06 | 0.47 
Denmark 0.25 |0.80 | 0.52 
England 0.54 |0.73 | 0.54 
Finland 0.67 |0.70 0.55 
France —0.02 |1.01 | 0.57 
Georgia —0.62 |1.25 |0.53 
Germany —0.13 0.97 | 0.53 
Hong Kong, SAR 1.10 |0.88 | 0.68 
Hungary —0.31 | 1.09 | 0.54 
Indonesia —0.01 | 1.10 | 0.60 


Iran, Islamic Republic of 5727 |2.14 |—0.07 |0.83 |0.54 ,—0.08 098 | 0.54 


Ireland 4415 | 2.27 0.00 |0.80 | 0.54 003 | 0.97 | 0.56 
Israel 4117 | 2.46 0.06 | 0.83 | 0.57 0.08 | 1.00 | 0.58 
Italy 4100 |2.17 |-0.01 |0.76 |0.52 |-0.07 |0.96 | 0.54 
Lithuania 4591 |2.17 |-0.02 |0.77 |0.52 |-0.04 |0.93 | 0.53 
Malta 3519 2.53 0.09 081 | 0.57 0.14 098 | 0.59 
Netherlands 3955 | 3.56 0.48 |0.72 | 0.61 052 |0.89 | 0.61 
New Zealand 5549 | 3.03 0.36 | 0.67 | 0.55 044 | 0.80 | 0.56 
Northern Ireland 3523 | 2.37 0.16 |0.61 | 0.46 0.17 | 0.75 | 0.47 
Norway 3112 | 2.50 0.19 |0.65 | 0.49 0.29 | 0.80 | 0.53 
Poland 4953 | 2.20 |-0.05 |0.82 |0.54 |-0.10 | 1.01 | 0.55 
Portugal 4037 |1.91 |-0.19 |0.83 |0.52 |-0.26 | 1.02 | 0.53 
Qatar 3947 | 2.82 0.17 | 0.89 | 0.62 0.19 | 1.08 | 0.62 
Romania 4592 |1.60 |-0.57 | 1.16 |0.57 |-0.71 |1.39 | 0.56 
Russian Federation 4444 |1.82 |-0.25 |0.86 |0.51 |-0.31 | 105 | 0.53 
Saudi Arabia 4425 | 2.55 0.06 |0.90 | 0.60 0.07 | 1.08 | 0.61 
Singapore 6275 | 4.25 0.66 | 0.74 | 0.65 077 |0.92 | 0.66 
Slovak Republic 5586 | 1.76 |-0.45 | 1.06 |0.56 |-0.57 | 1.29 | 0.56 
Slovenia 4456 | 2.13 |-0.02 |0.75 |0.51 |-0.02 |0.91 | 0.52 
Spain 8501 | 2.07 |-0.15 |0.88 |0.55 |-0.18 | 1.07 | 0.56 
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Table 4.4 (continued) 
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Sweden 


Trinidad and Tobago 0.54 |—0.75 | 1.34 0.55 
United Arab Emirates 0.58 |-0.14 | 1.14 | 0.59 
United States 0.60 0.20 |1.02 | 0.61 


Note N is the sample size, and X the observed mean score on the component. (0) is the estimated 
mean, o(0) is the standard deviation, and p is the estimated global reliability under the partial credit 


model (PCM) or the generalized partial credit model (GPCM) 


Table 4.5 Country characteristics component 5: school practices on parental involvement, school 


perspective 


PCM 


>< 


Country N 


GPCM 


(0) ei [p 


u(0) 


c(0) 


Azerbaijan, Republic of 169 | 32.89 0.32 |0.55 | 0.70 0.59 090 | 0.73 
Australia 269 | 35.41 0.64 |0.64 | 0.74 1.04 | 1.06 | 0.76 
Austria 158 | 31.00 0.05 |0.47 |0.64 |-0.14 081 |0.71 
Belgium (French) 118 | 23.37 |-0.80 |0.59 073 |-1.13 |0.92 | 0.74 
Bulgaria 147 | 30.14 |-0.04 |0.70 | 0.79 0.10 | 1.10 0.81 
Canada 1084 | 33.37 0.38 | 0.67 | 0.76 0.56 | 1.09 | 0.79 
Chinese Taipei 150 | 34.35 0.54 |0.88 | 0.83 077 | 1.48 0.85 
Colombia 149 | 32.73 035 |0.77 | 0.81 0.87 | 1.29 0.81 
Croatia 152 | 30.50 |-0.02 |0.46 | 0.63 0.01 |0.74 | 0.68 
Czech Republic 174 | 28.28 |-0.29 |0.46 |0.64 |-0.30 |0.81 | 0.72 
Denmark 231 | 25.93 |-0.54 |0.42 |0.60 |-1.03 |0.76 | 0.67 
England 120 | 32.83 029 0.57 | 0.71 0.30 |0.89 | 0.74 
Finland 139 | 25.60 |-0.59 |0.50 |0.68 |-1.00 |0.85 | 0.72 
France 167 | 27.52 |-0.35 |0.57 |0.73 ,—0.60 |0.99 | 0.78 
Georgia 171 | 30.85 0.07 |0.69 | 0.79 0.24 11417 | 0.82 
Germany 187 | 30.16 |-0.05 |0.50 |0.68 |-0.19 |0.82 | 0.72 
Hong Kong, SAR 125 | 30.18 |—-0.04 |0.63 |0.76 |—0.31 |1.05 | 0.80 
Hungary —0.14 | 0.85 | 0.73 
Indonesia —0.60 | 1.21 | 0.84 
Iran, Islamic Republic of 0.65 | 1.46 | 0.85 
Ireland —0.75 | 1.07 | 0.81 
Israel 0.31 [1.12 |0.81 
Italy —0.38 | 1.08 | 0.80 
Lithuania —0.07 | 0.83 | 0.72 
Malta 0.02 089 | 0.75 
Netherlands 117 | 26.97 |-0.42 |0.43 |0.61 "080 |0.77 | 0.69 
New Zealand 175 | 34.13 047 |0.57 | 0.70 0.56 |0.88 | 0.73 


(continued) 
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Table 4.5 (continued) 


Northern Ireland 


Norway 0.56 |—0.93 | 0.66 | 0.62 
Poland 0.69 0.22 |0.83 | 0.71 
Portugal 0.76 |-0.16 | 0.98 | 0.78 
Qatar 0.85 1.02 | 1.59 | 0.85 
Romania 0.78 0.76 |121 0.81 


0.62 0.69 0.82 | 0.70 
0.85 |—0.55 |1.43 0.88 


Russian Federation 
Saudi Arabia 


Singapore 0.73 0.31 |1.05 | 0.79 
Slovak Republic 0.72 |-0.20 |0.97 | 0.77 
Slovenia 0.656 | —-0.21 |0.81 | 0.71 
Spain 0.78 |-0.30 | 1.08 | 0.81 
Sweden 0.71 |-0.57 | 0.89 | 0.75 


Trinidad and Tobago 0.82 043 | 1.34 | 0.85 
United Arab Emirates 0.82 0.45 | 1.25 | 0.83 
United States 0.78 1.00 |1.29 |0.81 


Note N is the sample size, and X the observed mean score on the component. 4(0) is the estimated 
mean, o(0) is the standard deviation, and p is the estimated global reliability under the partial credit 
model (PCM) or the generalized partial credit model (GPCM) 


Table 4.6 Response frequencies and item parameter estimates under the generalized partial credit 
model for items in component 1: early literacy activities 


Item Slope Intercept K0) Relative frequency response 

categories 

CatO Cal Cat2 
ASBHO2A 1.26 1.84 0.44 0.54 0.41 0.05 
ASBHO2B 1.24 1.47 0.46 0.48 0.46 0.07 
ASBHO2C 0.77 0.98 0.23 0.49 0.41 0.11 
ASBHO2D 1.09 0.85 0.45 0.43 0.44 0.14 
ASBHO2E 0.95 1.80 0.24 0.62 0.34 0.04 
ASBHO2F 1.18 0.82 0.45 0.36 0.52 0.12 
ASBHO2G 1.24 0.57 0.52 0.33 0.51 0.16 
ASBHO02H 1.06 1.12 0.38 0.47 0.44 0.10 
ASBHO?2I 1.07 0.89 0.43 0.44 0.42 0.13 
Note The latent distributions of the countries are normed to an overall mean of zero. Slope and 
intercept are the parameters aiO and the mean of the location parameters bil, bi2, ..., bih, etc., 


respectively. /(0) is the information value of the item at 0 = 0. Cam, Catl, Cat2 indicate the 
frequency with which item categories 0, 1 and 2 are endorsed, respectively. The components, items 
and corresponding category labels are described in Table 3.2 
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Table 4.7 Response frequencies and item parameter estimates under the generalized partial credit 
model for items in component 2: help with homework 


Item Slope Intercept K0) Relative frequency response 

categories 

Cam Cal Cat2 Cat3 
ASBHO9A 1.17 2.44 0.21 0.78 0.18 0.03 0.02 
ASBHO9B 1.63 2.01 0.78 0.56 0.32 0.07 0.05 
ASBH09C 1.15 2.15 0.18 0.81 0.14 0.03 0.03 
ASBHO9D 1.10 2.27 0.24 0.73 0.22 0.03 0.02 
ASBH09E 1.56 2.51 0.31 0.77 0.17 0.03 0.04 
ASBH09F 1.69 1.09 1.25 0.43 0.33 0.10 0.14 
ASBH09G 2.26 1.87 1.62 0.43 0.37 0.12 0.08 
ASBH09H 1.45 1.66 0.77 0.47 0.38 0.11 0.04 
Note The latent distributions of the countries are normed to an overall mean of zero. Slope and 
intercept are the parameters ai0 and the mean of the location parameters bil, bi2, ..., bih, etc., 


respectively. /(0) is the information value of the item at 0 = 0. Cat0, Cat), Cat2, and Cat3 indicate 
the frequency with which item categories 0, 1, 2 and 3 are endorsed, respectively. The content of 
the components, items and corresponding category labels are described in Table 3.2 


Table 4.8 Response frequencies and item parameter estimates under the generalized partial credit 
model for items in component 3: school practices on parental involvement, parent perspective 


Item Slope Intercept K0) Relative frequency response 

categories 

Cam Cal Cat2 Cat3 
ASBHI0A 0.61 1.41 0.15 0.54 0.37 0.07 0.02 
ASBH10B 0.61 0.36 0.34 0.30 0.31 0.23 0.16 
ASBHI0E 0.58 0.52 0.30 0.38 0.29 0.19 0.14 
Note The latent distributions of the countries are normed to an overall mean of zero. Slope and 
intercept are the parameters ai0 and the mean of the location parameters bil, bi2, ..., bih, etc., 


respectively. /(0) is the information value of the item at 0 = 0. Cat0, Cat), Cat2, and Cat3 indicate 
the frequency with which item categories 0, 1, 2 and 3 are endorsed, respectively. The 
components, items and corresponding category labels are described in Table 3.2 


value for PIRLS item ASBHO2A (“read books"), indicates that this item of the scale 
performed best in this respect. Local reliability is further supported if the item 
location parameters agree closely with the mean of a latent distribution. In this 
respect, item ASBH02G (“play word games”) performed best, because the latent 
distributions of the countries were normed to an overall mean of zero. Together the 
intercept and slope parameters determine the information value of an item. Higher 
values for the information value of an item at 0 = 0, namely I(0), indicate the item 
made a higher contribution to the local reliability of the component. 

For component 1 (early literacy activities), the item ASBH02C (“sing songs”) 
has a lower information value than the other items. This should be taken into 
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Table 4.9 Response frequencies and item parameter estimates under the generalized partial credit 
model for items in component 4: student perception of parental involvement 


Item Slope Intercept K0) Relative frequency response 

categories 

Cam Cal Cat2 Cat3 
ASBG07A 1.01 1.47 0.32 0.67 0.21 0.05 0.07 
ASBG07B 0.96 1.15 0.43 0.56 0.27 0.08 0.09 
ASBG07C 0.85 1.35 0.22 0.75 0.14 0.04 0.08 
ASBG07D 0.77 1.21 0.22 0.73 0.14 0.04 0.09 
ASBR09C 0.55 1.55 0.09 0.76 0.18 0.04 0.02 
Note The latent distributions of the countries are normed to an overall mean of zero. Slope and 
intercept are the parameters ai0 and the mean of the location parameters bil, bi2, ..., bih, etc., 


respectively. /(0) is the information value of the item at 0 = 0. Cat0, Cat), Cat2, and Cat3 indicate 
the frequency with which item categories 0, 1, 2 and 3 are endorsed, respectively. The 
components, items and corresponding category labels are described in Table 3.2 


Table 4.10 Response frequencies and item parameter estimates under the generalized partial 
credit model for items in component 5: school practices on parental involvement, school 
perspective 


Item Slope Intercept HM Relative frequency response categories 
Ca Cal Cat2 Cat3 Cat4 
ACBGIIAA 0.75 —2.88 0.14 0.00 0.01 0.37 0.62 - 
ACBGIIAB 0.91 -2.95 0.20 0.00 0.02 0.33 0.65 = 
ACBG11AC 0.87 —2.34 0.23 0.00 0.05 0.39 0.56 = 
ACBGIIAD 0.57 —1.34 0.14 0.03 0.07 0.29 0.62 = 
ACBGIIBA 0.47 —0.66 0.16 0.07 0.16 0.37 0.40 = 
ACBGIIBB 0.51 —0.64 0.20 0.06 0.30 0.32 0.32 - 
ACBGIICA 0.70 —0.55 0.29 0.05 0.33 0.38 0.23 = 
ACBG11CB 0.84 =1:29 0.38 0.03 0.18 0.35 0.44 = 
ACBG11CC 1.27 =127 0.72 0.01 0.38 0.37 0.24 _ 
ACBGIICD 1.13 —1.42 0.66 0.01 0.45 0.29 0.25 = 
ACBGIICE 1.09 —1.10 0.60 0.03 0.30 0.39 0.29 _ 
ACBGIICF 0.41 0.02 0.18 0.23 0.25 0.27 0.25 - 
ACBGIICG 0.52 0.26 0.23 0.24 0.31 0.30 0.15 - 
ACBG12E 0.25 —0.35 0.05 0.02 0.13 0.46 0.31 0.09 
ACBGI2F 0.20 —0.18 0.03 0.04 0.17 0.46 0.26 0.08 
Note The latent distributions of the countries are normed to an overall mean of zero. Slope and 
intercept are the parameters ai0 and the mean of the location parameters bil, bi2, ..., bih, etc., 


respectively. /(0) is the information value of the item at 0 = 0. CatO, Cal, Cat, Cat3, and Cat4 
indicate the frequency with which item categories 0, 1, 2, 3 and 4 are endorsed, respectively. The 
components, items and corresponding category labels are described in Table 3.2 
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account when redesigning the instrument for future surveys; in other words, this 
item may be the first candidate for replacement. Compared to component | (early 
literacy activities), the items in component 2 (helping with homework) were more 
informative, while items in component 3 (school practices on parental involvement, 
parent perspective) performed poorly. Components 4 (school practices for parental 
involvement from a student perspective) and 5 (school practices for parental 
involvement from a school perspective) provided differing results; in particular, the 
last two items of component 5 (“parental support for student achievement within 
school” and “parental involvement in school activities”) performed particularly 
poorly. 

Comparing the parameter estimates in the GPCM and the GPCM with random 
item parameters (henceforth the random GPCM) revealed that the agreement 
between the slopes and intercepts under the GPCM and the means of the slopes and 
intercepts under the random GPCM was high (Tables 4.11, 4.12, 4.13, 4.14 and 
4.15). A higher variance provides an initial indication that the item functions dif- 
ferently in different countries, a topic we address in more detail later. Here, the 
effects are global over countries and thus only permit global inferences. For 
instance, for component 1, the last item, ASBH02I (“read aloud signs and tables") 
has the lowest CDIF because the variance of the intercepts and slopes across the 
countries is the lowest among the items (Table 4.11). A low variance indicates that 
the item parameters do not vary much across countries. Evaluating the relative 
CDIF of the other eight items is more difficult, because of the trade-off between the 
standard deviation for the slope and the intercept. 


Table 4.11 Item parameter estimates under the generalized partial credit model (GPCM) and 
GPCM with random item parameters for items in component 1: early literacy activities 


Item GPCM GPCM random item parameters 
Slope Intercept Slope SD (Slope) Intercept SD (Intercept) 

ASBH02A 1.26 1.84 1.37 0.22 2.06 0.66 
ASBHO2B 1.24 1.47 1.25 0.15 1.50 0.31 
ASBHO2C 0.77 0.98 0.80 0.12 1.03 0.34 
ASBH02D 1.09 0.85 1.21 0.18 0.86 0.44 
ASBH02E 0.95 1.80 1.01 0.19 2.01 0.68 
ASBH02F 1.18 0.82 1.33 0.23 0.93 0.42 
ASBH02G 1.24 0.57 1.35 0.15 0.60 0.27 
ASBH02H 1.06 1.12 1.16 0.16 1.17 0.41 
ASBHO2ZI 1.07 0.89 1.09 0.11 0.87 0.22 


Note The latent distributions of the countries are normed to an overall mean of zero. SD (Slope) 
indicates the standard deviation of the slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 
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Table 4.12 Item parameter estimates under the generalized partial credit model (GPCM) and 
GPCM with random item parameters for items in component 2: help with homework 


Item GPCM GPCM random item parameters 
Slope Intercept Slope SD (Slope) Intercept SD (Intercept) 

ASBH09A 1.17 2.44 1.331 0.619 3.686 1.547 
ASBH09B 1.63 2.01 1.313 0.534 2.947 1.880 
ASBH09C 1.15 2.15 1.396 0.554 2.199 1.203 
ASBH09D 1.10 2.27 1.227 0.314 3.736 1.610 
ASBH09E 1.56 2.51 1.437 0.634 3.446 1.208 
ASBH09F 1.69 1.09 1.477 0.503 0.707 1.251 
ASBH09G 2.26 1.87 1.308 0.434 0.796 1.154 
ASBH09H 1.45 1.66 1.559 0.224 1.518 1.210 


Note The latent distributions of the countries are normed to an overall mean of zero. SD (Slope) 
indicates the standard deviation of the slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 


Table 4.13 Item parameter estimates under the generalized partial credit model (GPCM) and 
GPCM with random item parameters for items in component 3: school practices on parental 
involvement, parent perspective 


Item GPCM GPCM random item parameters 

Slope Intercept Slope SD (Slope) Intercept SD (Intercept) 
ASBHI0A 0.61 1.41 1.218 1.388 4.477 4.172 
ASBH10B 0.61 0.36 4.144 1.601 2.751 4.923 
ASBH10E 0.58 0.52 3.843 1.791 3.469 5.232 


Note The latent distributions of the countries are normed to an overall mean of zero. SD (Slope) 
indicates the standard deviation of the slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 


Table 4.14 Item parameter estimates under the generalized partial credit model (GPCM) and 
GPCM with random item parameters for items in component 4: student perception of parental 
involvement 


Item GPCM GPCM random item parameters 

Slope Intercept Slope SD (Slope) Intercept SD (Intercept) 
ASBG07A 1.01 1.47 0.924 0.161 1.473 1.102 
ASBG07B 0.96 1.15 0.994 0.357 1.155 0.943 
ASBG07C 0.85 1.35 0.989 0.316 1.937 2.614 
ASBG07D 0.77 1.21 0.990 0.240 1.917 3.017 
ASBR09C 0.55 1.55 0.553 0.050 2.100 2.782 


Note The latent distributions of the countries are normed to an overall mean of zero. SD (Slope) 
indicates the standard deviation of the slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 
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Table 4.15 Item parameter estimates under the generalized partial credit model (GPCM) and 
GPCM with random item parameters for items in component 5: school practices on parental 
involvement, school perspective 


Item GPCM GPCM random item parameters 
Slope Intercept Slope SD (Slope) Intercept SD (Intercept) 

ACBGIIAA ` | 0.75 —2.88 0.689 0.664 —1.667 1.396 
ACBGIIAB 0.91 —2.95 1.029 0.377 —2.122 0.797 
ACBGIIAC 0.87 —2.34 0.998 0.506 —2.110 0.778 
ACBGIIAD _ | 0.57 —1.34 0.466 1.042 —1.480 0.461 
ACBGIIBA 0.47 —0.66 0.645 0.876 —0.581 1.033 
ACBGIIBB 0.51 —0.64 0.627 0.807 —0.583 0.462 
ACBGIICA 0.70 —0.55 0.887 0.491 —0.576 0.434 
ACBGIICB 0.84 —1.29 0.890 | 0.621 —1.120 0.614 
ACBGIICC 1.27 =1.27 1.236 | 0.620 —0.995 0.682 
ACBGIICD 1.13 —1.42 1.4904 | 0.515 —1.122 0.625 
ACBGIICE 1.09 —1.10 1.132 0.229 —1.023 0.168 
ACBGIICF 0.41 0.02 0.548 0.738 0.029 0.342 
ACBGIICG 0.52 0.26 0.737 0.514 0.071 0.781 
ACBGI2B 0.25 —0.35 0.123 1.453 0.551 1.954 
ACBGI2F 0.20 —0.18 0.279 1.431 —0.030 1.789 


Note The latent distributions of the countries are normed to an overall mean of zero. SD (Slope) 
indicates the standard deviation of the slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 


This pattern is repeated for component 2; the items ASBHO9F (“helping child 
practice reading") and ASBHO9G (“helping child practice math skills") performed 
slightly better than the other items (Table 4.12). Conversely, component 3 showed a 
substantial difference between the item parameters estimated with the GPCM and 
those estimated using the random GPCM (Table 4.13), indicating this short scale 
was quite unstable. 

The analyses of components 4 and 5 indicated all the items performed compa- 
rably with respect to CDIF (Tables 4.14 and 4.15), although questions surrounding 
specific item-by-country interaction and the influence of the inferences on country 
means and latent regression remain unanswered. 

We compared CDIF as identified by the random GPCM with CDIF as identified 
using the latent residuals defined by Eq. (4.3) and aggregated over countries 
(Tables 4.16, 4.17, 4.18, 4.19 and 4.20). Overall the agreement between the 
methods was high. For instance, item ASBHO?I performed strongly in all methods, 
as did item ASBHO2G (Table 4.16). In general, the residuals with the GPCM are 
smaller than those with the PCM, because the latter model has fewer parameters. 
Other studies (see e.g., Glas and Jehangir 2014) confirm this expectation. However, 
we found that differences between the PCM and the GPCM were very small. We 
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Table 4.16 Absolute differential item functioning (DIF) under the partial credit model (PCM) and 
the generalized partial credit model (GPCM) and standard deviation random item parameters on 
items in component 1: early literacy activities 


Item PCM GPCM SD (Slope) SD (Intercept) 
ASBH02A 0.12 0.11 0.228 0.667 
ASBHO2B 0.08 0.08 0.158 0.318 
ASBHO2C 0.09 0.10 0.126 0.349 
ASBHO2D 0.12 0.12 0.183 0.443 
ASBHO2E 0.10 0.10 0.192 0.688 
ASBHO2F 0.09 0.09 0.239 0.421 
ASBH02G 0.07 0.07 0.155 0.279 
ASBHO2H 0.10 0.10 0.161 0.416 
ASBHO?I 0.07 0.07 0.112 0.229 


Note The columns labeled PCM and GPCM give the mean residuals as estimated under the 
unidimensional versions of these two models. SD (Slope) indicates the standard deviation of the 
slope. SD (Intercept) indicates the standard deviation of the intercept. Item descriptions are 
provided in Table 3.2 


Table 4.17 Absolute differential item functioning (DIF) under the partial credit model (PCM) and 
the generalized partial credit model (GPCM) and standard deviation random item parameters on 
items in component 2: help with homework 


Item PCM GPCM SD (Slope) SD (Intercept) 
ASBH09A 0.11 0.12 0.619 1.547 
ASBHO9B 0.07 0.07 0.534 1.880 
ASBHO09C 0.10 0.10 0.554 1.203 
ASBHO09D 0.10 0.10 0.314 1.610 
ASBHO9E 0.08 0.08 0.634 1.208 
ASBHO9F 0.14 0.12 0.503 1.251 
ASBH09G 0.08 0.06 0.434 1.154 
ASBH09H 0.07 0.07 0.224 1.210 


Note The columns labeled PCM and GPCM give the mean residuals as estimated under the 
unidimensional versions of these two models. SD (Slope) indicates the standard deviation of the 
slope. SD (Intercept) indicates the standard deviation of the intercept. Item descriptions are 
provided in Table 3.2 


Table 4.18 Absolute differential item functioning (DIF) under the partial credit model (PCM) and 
the generalized partial credit model (GPCM) and standard deviation random item parameters on 
items in component 3: school practices on parental involvement, parent perspective 


Item PCM GPCM SD (Slope) SD (Intercept) 
ASBHIOA 0.13 0.47 1.388 4.172 
ASBHIOB 0.07 0.36 1.601 4.923 
ASBHIOE 0.09 0.38 1.791 5.232 


Note The columns labeled PCM and GPCM give the mean residuals as estimated under the 
unidimensional versions of these two models. SD (Slope) indicates the standard deviation of the 
slope. SD (Intercept) indicates the standard deviation of the intercept. Item descriptions are 
provided in Table 3.2 
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Table 4.19 Absolute 
differential item functioning 
(DIF) under the partial credit 
model (PCM) and the 
generalized partial credit 
model (GPCM) and standard 
deviation random item 
parameters on items in 
component 4: student 
perception of parental 
involvement 


Table 4.20 Absolute 
differential item functioning 
(DIF) under the partial credit 
model (PCM) and the 
generalized partial credit 
model (GPCM) and standard 
deviation random item 
parameters on items in 
component 5: school practices 
on parental involvement, 
school perspective 


51 
Item PCM | GPCM SD SD 
(Slope) (Intercept) 

ASBGO7A (0.08 0.07 0.161 1.102 
ASBGOT7B 0.09 0.08 0.357 0.943 
ASBGO07C 0.07 0.08 0.316 2.614 
ASBGO7D  |0.12 0.12 0.240 3.017 
ASBRO9C 0.07 0.08 0.050 2.782 


Note The columns labeled PCM and GPCM give the mean 
residuals as estimated under the unidimensional versions of these 
two models. SD (Slope) indicates the standard deviation of the 
slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 


Item PCM GPCM SD SD 
(Slope) (Intercept) 

ACBGIIAA ` | 0.23 0.21 0.664 1.396 
ACBGIIAB 0.19 0.17 0.377 0.797 
ACBGIIAC 0.17 0.16 0.506 0.778 
ACBGIIAD 0.16 0.16 1.042 0.461 
ACBGIIBA 0.32 0.35 0.876 1.033 
ACBGIIBB 0.24 0.24 0.807 0.462 
ACBGIICA 0.20 0.18 0.491 0.434 
ACBGIICB 0.22 0.23 0.621 0.614 
ACBGIICC 0.15 0.13 0.620 0.682 
ACBGIICD 0.21 0.17 0.515 0.625 
ACBGIICE 0.11 0.11 0.229 0.168 
ACBGIICF 0.29 0.32 0.738 0.342 
ACBGIICG 0.32 0.34 0.514 0.781 
ACBGI2E 0.26 0.27 1.453 1.954 
ACBGI2F 0.25 0.24 1.431 1.789 


Note The columns labeled PCM and GPCM give the mean 
residuals as estimated under the unidimensional versions of these 
two models. SD (Slope) indicates the standard deviation of the 
slope. SD (Intercept) indicates the standard deviation of the 
intercept. Item descriptions are provided in Table 3.2 


tentatively conclude the PCM fits the data quite well. A striking exception, again, 
was component 3. Here the fit of the GPCM was worse than the fit of the PCM, 
which leads to the conclusion that the slopes are very hard to estimate. This is in 
agreement with the reported low global reliability. Obviously, variance in the 6- 
distribution is too small to support a proper estimate of the slope parameters. 
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Table 4.23 Residual analysis for country-by-item interactions for component 3: school practices 
on parental involvement, parent perspective 


Country Item 10 % 20 96 Absolute residual 
1 2 3 CDIF CDIF 
Azerbaijan, Republic of + |0 1 0.084 
Australia 0 0 0.032 
Austria ++ 1 1 0.102 
Belgium (French) + 0 1 0.088 
Bulgaria + 0 1 0.110 
Canada 0 0 0.058 
Chinese Taipei 0 0 0.057 
Colombia 1 1 0.112 
Croatia 1 1 0.090 
Czech Republic 1 1 0.085 
Denmark 1 1 0.071 
Finland 1 1 0.096 
France 0 1 0.081 
Georgia 1 1 0.088 
Germany 1 1 0.164 
Hong Kong, SAR 0 1 0.054 
Hungary 0 0 0.026 
Indonesia 0 2 0.142 
Iran, Islamic Republic of 0 0 0.034 
Ireland 0 0 0.073 
Israel 0 0 0.042 
Italy 0 1 0.106 
Lithuania 0 0 0.029 
Malta 1 1 0.082 
Netherlands 0 0 0.039 
New Zealand 0 0 0.037 
Northern Ireland 0 0 0.030 
Norway 0 0 0.104 
Poland 0 0 0.050 
Portugal 0 0 0.037 
Qatar 0 1 0.075 
Romania 1 1 0.127 
Russian Federation 0 1 0.088 
Saudi Arabia 0 1 0.048 
Singapore 1 1 0.083 
Slovak Republic 0 0 0.049 
Slovenia ++ 1 1 0.072 
Spain 0 0 0.018 


(continued) 
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Table 4.23 (continued) 


Country 20 % Absolute residual 
CDIF 

Sweden 0 0.016 

Trinidad and Tobago - 0 1 0.109 

United Arab Emirates 0 0 0.044 


Note + indicates that residual belongs to the 20 % most positive residuals, ++ indicates that 
residual even belongs to the 10 % most positive residuals. — indicates that residual belongs to the 
20 % most negative residuals, —— indicates that residual even belongs to the 10 % most negative 
residuals. The 10 % cultural differential item functioning (CDIF) and 20 % CDIF columns give the 
number of outliers in the two respective regions. Absolute residual refers to the means over items 
of the absolute values of the residuals. Item descriptions are provided in Table 3.2 


Table 4.24 Residual analysis for country-by-item interactions for component 4: student 
perception of parental involvement 


Country Item 10 % 20 % Absolute 
1 |2 3 4 5 CDIF CDIF residual 

Azerbaijan, Republic of 0 0 0.040 
Australia 0 0 0.060 
Austria 0 0 0.037 
Belgium (French) _ 0 1 0.076 
Bulgaria + 0 1 0.075 
Canada 0 0 0.068 
Chinese Taipei ++ == 2 2 0.117 
Colombia 0 0 0.034 
Croatia _ ++ 1 2 0.094 
Czech Republic 0 0 0.051 
Denmark 0 0 0.056 
England + _ 0 2 0.088 
Finland ++ |1 1 0.103 
France 0 0 0.068 
Georgia + 0 1 0.075 
Germany + == 1 2 0.146 
Hong Kong, SAR _ 0 1 0.087 
Hungary _ ++ 1 2 0.110 
Indonesia esses [rp 1 0.080 
Iran, Islamic Republic + 0 1 0.071 
of 

Ireland + m 1 2 0.112 


(continued) 
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Table 4.24 (continued) 
Country Item 10 % 20 % Absolute 
1 2 3 4 5 CDIF CDIF residual 
Israel ++ 1 1 0.120 
Italy + 0 1 0.066 
Lithuania 0 0 0.061 
Malta _ 0 1 0.078 
Netherlands —— |t [t+ |+ |4 4 0.233 
New Zealand 0 0 0.027 
Northern Ireland + |+ — |- 2 4 0.197 
Norway + 0 1 0.087 
Poland + — | |2 3 0.158 
Portugal + 0 1 0.074 
Qatar + 0 1 0.089 
Romania ++ 1 1 0.082 
Russian Federation 0 0 0.073 
Saudi Arabia _ ++ 1 2 0.133 
Singapore ge ++ |2 2 0.104 
Slovak Republic 0 0 0.070 
Slovenia 0 0 0.059 
Spain 0 0 0.056 
Sweden + == 1 2 0.090 
Trinidad and Tobago 0 0 0.049 
United Arab Emirates 0 0 0.060 
United States 0 0 0.084 


Note + indicates that residual belongs to the 20 % most positive residuals, ++ indicates that residual even 
belongs to the 10 % most positive residuals. — indicates that residual belongs to the 20 % most negative 
residuals, —— indicates that residual even belongs to the 10 % most negative residuals. The 10 % cultural 
differential item functioning (CDIF) and 20 % CDIF columns give the number of outliers in the two 
respective regions. Absolute residual refers to the means over items of the absolute values of the 
residuals. Item descriptions are provided in Table 3.2 


We then addressed the distribution of country-by-item interaction across coun- 
tries and items, to determine whether the sizes and directions of the residuals were 
randomly distributed across all countries and items, or whether they exhibited 
notable patterns of interaction (Tables 4.21, 4.22, 4.23, 4.24 and 4.25). Residuals 
were defined by Eq. (4.3), estimated under the GPCM, and calculated for every 
country, with that country as a focus and all other countries as a reference. To 
simplify, here we shall not consider the specific values of the residuals, but instead 
concentrate on the outlying values. For example, if we examine results obtained for 
the Republic of Azerbaijan and Australia for component | (early literacy activities, 
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Table 4.21), it is clear that, aggregated over the items, the mean absolute residual 
for the Republic of Azerbaijan is much larger than the mean absolute residual for 
Australia. The responses were coded 0, 1 and 2, so the residuals, which are the 
differences between a mean observed and expected response are also on a scale 
from 0 to 2. Closer inspection at the item level for Republic of Azerbaijan reveals 
that items 3 and 5 have residuals among the 10 % most positive among the 
countries, while the items 6 and 8 have residuals among the 10 % most negative 
among the countries. Australia, however, has only one negative residual, and this is 
among the 20 % most negative residuals among the countries. Checking the 
absolute residuals further reveals Poland fits the model best with the lowest CDIF, 
while Indonesia has the most significant CDIF. 

In a similar way, component 2 (helping with homework) functions very differ- 
ently in the Netherlands than in other countries (Table 4.22), probably because 
giving students homework is not a daily practice in Dutch primary schools. This 
different item functioning is indicated by both the high mean for the absolute values 
of the residuals and the large number of outliers among the residuals. Canada fits 
the model best, having the lowest CDIF for this component. For component 3 
(school practices on parental involvement, parents perspective) the highest mean 
absolute residual was found for Germany. However, the scale for measuring school 
practices on parental involvement from the school perspective (component 5) 
showed relatively little evidence of CDIF. 

We undertook a marginal count of the outliers for the items aggregated over the 
countries (Table 4.26). No one item count was prominent, although the first item in 
component 3 (“my child’s school includes me in my child’s education”) seemed 
more susceptible to CDIF than other items, since this item had the greatest number 
of residual outliers among countries: 13 in the 10 % outliers region and 15 in the 
20 % outliers region. Items 5 (“volunteering”) and 13 (“organize workshops or 
seminars for parents on learning or pedagogical issues”) within component 5 also 
scored more highly than other items in the component. However, this does not of 
course mean that these items have CDIF; if 10 and 20 % extreme values are 
considered, then 10 and 20 % of the residuals must be included, thus such infor- 
mation only serves as a tool to further scrutinize the items. 

We also calculated country-specific factor loadings for the bi-factor model, 
where we first transformed country-specific factor loadings to standard normals, 
and then identified the 2.5 and 5 % most extreme outlying values (Tables 4.27, 
4.28, 4.29, 4.30 and 4.31). This distribution of country-specific factor loadings 
gives an indication of the extent to which items load on a country-specific factor in 
addition to the general factor of the item, and can, as in our earlier residual analysis, 
be used to determine whether the sizes and directions of the factor loadings are 
randomly distributed across all countries and items, or whether they exhibit notable 
patterns of interaction. 

For component 1, the greatest number of outliers of the country-specific factor 
loadings and the highest mean absolute factor loading were found for Colombia 
(Table 4.27), suggesting a high level of CDIF. Interestingly, in the residual analysis 
for this component, a total of 15 countries showed a higher mean absolute residual 
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Table 4.29 Outliers of country-specific factor loadings in the bi-factor model for component 3: 
school practices on parental involvement, parent perspective 


Country Item 2.5 96 5 96 Mean absolute loading 
1 |2 Outlier Outlier 
Azerbaijan, Republic of + |++ 1 2 1.097 
Australia 0 0 0.293 
Austria 0 0 0.203 
Belgium (French) 0 0 0.223 
Bulgaria 0 0 0.262 
Canada 0 0 0.423 
Chinese Taipei 1 1 0.640 
Colombia 0 0 0.159 
Croatia 0 0 0.194 
Czech Republic 0 0 0.393 
Denmark 0 0 0.284 
Finland 0 0 0.293 
France 0 0 0.293 
Georgia 0 0 0.240 
Germany 0 0 0.409 
Hong Kong, SAR 0 0 0.168 
Hungary 1 1 1.521 
Indonesia 0 1 0.500 
Iran, Islamic Republic of 0 0 0.362 
Ireland 0 0 0.279 
Israel 0 0 0.216 
Italy 0 0 0.131 
Lithuania 0 0 0.174 
Malta 0 0 0.418 
Netherlands 0 0 0.331 
New Zealand 0 0 0.260 
Northern Ireland 0 0 0.321 
Norway 0 0 0.228 
Poland 0 0 0.213 
Portugal 0 0 0.205 
Qatar 0 0 0.297 
Romania 0 0 0.430 
Russian Federation 0 0 0.153 
Saudi Arabia 0 0 0.184 
Singapore 0 0 0.150 
Slovak Republic 0 0 0.180 
Slovenia 0 0 0.228 


(continued) 


68 


Table 4.29 (continued) 


4 Modeling Parental Involvement 


Country 2.5 96 Mean absolute loading 
Outlier Outlier 

Spain 0.347 

Sweden 0 0 0.315 

Trinidad and Tobago 0 0 0.517 

United Arab Emirates 0 0 0.175 


Note + indicates factor loading belongs to the 5 % most positive loading, ++ indicates factor 
loading belongs to the 2.5 % most positive loading. — indicates factor loading belongs to the 5 96 
most negative loading, —— indicates factor loading belongs to the 2.5 % most negative loading. 
The 2.5 % cultural differential item functioning (CDIF) and 5 % CDIF columns give the number of 
outliers in the two respective regions. Mean absolute loading refers to the means over items of the 
absolute values of country-specific factor loadings. Item descriptions are provided in Table 3.2 


Table 4.30 Outliers of country-specific factor loadings in the bi-factor model for component 4: 
student perception of parental involvement 


Country Item 2.5 96 5 96 Mean absolute 
1 2 3 4 5 Outlier | Outlier | loading 

Azerbaijan, Republic 0 0 0.024 
of 

Australia 0 0 0.012 
Austria 0 0 0.021 
Belgium (French) + 1 1 0.026 
Bulgaria 0 0 0.016 
Canada + ++ 1 2 0.048 
Chinese Taipei 0 0 0.012 
Colombia ER = 1 2 0.044 
Croatia _ == |== qae 3 4 0.084 
Czech Republic 0 0 0.018 
Denmark 0 0 0.010 
England 0 0 0.027 
Finland _ 0 1 0.038 
France _ _ 0 H 0.035 
Georgia ++ |++ |+ ++ 1 4 0.057 
Germany == 1 1 0.034 
Hong Kong, SAR 0 0 0.011 
Hungary 0 0 0.026 
Indonesia _ 0 1 0.029 
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Table 4.30 (continued) 
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Country Item 2.5 96 5 96 Mean absolute 
1 3 Outlier | Outlier | loading 

Iran, Islamic Republic + 1 1 0.034 
of 

Ireland _ 0 1 0.022 
Israel 0 0 0.024 
Italy 0 0 0.023 
Lithuania ++ |+ 2 3 0.052 
Malta 0 0 0.008 
Netherlands _ == 1 2 0.048 
New Zealand 0 0 0.031 
Northern Ireland 0 1 0.038 
Norway 0 0 0.021 
Poland ++ 0 1 0.025 
Portugal _ 0 1 0.030 
Qatar ++ 0 1 0.034 
Romania ++ 0 1 0.037 
Russian Federation 0 0 0.027 
Saudi Arabia _ 0 1 0.028 
Singapore 0 0 0.031 
Slovak Republic 0 0 0.016 
Slovenia 0 0 0.018 
Spain 0 0 0.020 
Sweden 0 0 0.022 
Trinidad and Tobago 0 0 0.020 
United Arab Emirates == |== 2 2 0.047 
United States 0 0 0.019 


Note + indicates factor loading belongs to the 5 % most positive loading, ++ indicates factor 
loading belongs to the 2.5 % most positive loading. — indicates factor loading belongs to the 5 % 
most negative loading, —— indicates factor loading belongs to the 2.5 % most negative loading. 
The 2.5 96 cultural differential item functioning (CDIF) and 5 % CDIF columns give the counts of 
the outliers in the two respective regions. Mean absolute loading refers to the means over items of 
the absolute values of country-specific factor loadings. Item descriptions are provided in Table 3.2 


4 Modeling Parental Involvement 


(penunuoo) 
0£0'0 0 0 ATI 
cLO'O T T + ist 
0s0°0 0 0 purely 
SOU 0 0 uel] 
6$0°0 0 0 eisouopug 
0L0°0 0 0 Aresuny 
1v0°0 0 0 suoy SuoH 
09070 I I + Kutullor) 
Lv0'0 I 0 2131090 
8c0'0 0 0 oouelq 
990'0 I I pueru 
1v0'0 0 0 purlsuq 
100 0 0 xreuiuoqq 
Lv0'0 0 0 oygnday u2oz;) 
e700 0 0 eneo 
990'0 I I €IQUIO[O) 
Tcro T T + Todre l, əƏsəurtuo 
LS0'0 0 0 tpeuv;) 
ESOO 0 0 eues[ng 
¢L0°0 [4 T + (D uiniS[og 
8700 0 0 einsny 
8L0'0 I I ei[ensny 
£v0'0 0 0 uefieq1ozy 
Jong} mno, SI| HO €f} cl) II | OI I 
Surpeoy ejnjosqe uvoj %S\) BST ua Anunog 


eAnoedsied [oouos *jueureA[oAur [e3uared uo soonovid [oouos :¢ JUAUOdUIOD JO} [əpout 1032vj-1q ou ur SSUIpRO] 107927 DYTOIds-ANUNOD Jo SIMO Te€'p AQEL 


71 


42 Results of Modeling Country-Specific Differential Item Functioning 


TE APL ur popraoid ore suonduosop WAJ `s8utpeo[ 107327 ogr»eds-Anunoo Jo sən[eA ojn[osqe ou JO SWAI 
JOAO SUBOU ƏY) 0} s1ojor Surpeo[ ojnposqe ULIN “SUOTSaI ƏAn59dsƏ1 OA} OU} ut SIJOTPNO Jo 1oquinu ou) SAIS suumgoo JIAO % ç pue (JIAO) Suruonounj war 
[enuo1ogrp [eIm[nə % cc eu `Suipeo[ eAnesou Jsou % c'c ou o) s8uo|əq Surpeoj 10792} sarpur —— “Burtpeo| AA DEZaU Isou % c I) o) SSUOTIq Surpeoj Jop) 
sojeorpur — “SUIPeOT eAnisod 3soui % cc ou oj s8šuo]əq Surpeo[ 10738} sojeorpur ++ “Surpeo[ oAnrsod Isour % ç I o) s8uo[əq SUIPLOT 10798} sojeorpur + AIO0N 


£900 0 0 SIIS pouf) 
FSOO 0 0 save” qery Daun 
690'0 I I osego], pue pepun, 
1400 0 0 uəpəAçs 
vr0'0 0 0 ureds 
£90'0 I 0 ++ PIUƏAO|S 
KFOO 0 0 oiqndos yeao[s 
Ss0'0 T 0 ++| ++ e1odesurg 
Or0'0 0 0 viqely Ipnes 
L90'0 [4 T + + 'pe4 uvissny 
CPOO 0 0 DIUPUIOT 
Tr0'0 0 0 PÒ 
6LVO r r + + jesmiog 
EITO € £ + + puejod 
£80'0 I 0 ++ KO MION 
SOTO € € + + DUEIaI UNON 
9IT0 C T + + pue[poZ MƏN 
100 0 0 spue[rouoN 
8800 T T BYP 
ce0'0 0 0 prupnu)rT 
Jong} mno, St} HO €I cl) It} OI C I 
Surpeo| ojnjosqe Uew %ç| AST at Ámunoo 


(penunuoo) Ier AQEL 


4 Modeling Parental Involvement 


72 


T'E ILL ur pequosop oie sjaqey L10330 Surpuodsərio2 pue sau *sjuouoduroo ou], ‘ssurpeoT 
10792} JO uornqungsip au Jo UOIBAI % ç Əutənxə pue % cz ƏutƏənxə ou) ur sr Surpeo[ 107927 ojroeds-Anunoo əu JO 191M0 ue sour jo 1equinu əu jo juno;) 2/0N 


0 0 I 0 0 T € 0 0 [4 I € 0I OI D 
I IT 6 9 9 Y 
€ I I € 
CT Ic 0c T 9 I c SI [4 
VI Or SI 8I €T I 8 6T I % € 
0 0 0 0 0 0 € 0 0 ré I € 6 6 ç 
0 9 € [4 € r 
T 0 I € 
I I DI T 
Y 9 d I % ŞT 
SI val €I cl II OI 8 L 9 ç Y € [4 I 
Sat juəouoduioo uor8əq 


JuəuməA[oAut [vjuoed UO SWA! SSOIOR [opour 10}9eJ-1q au ur SSUTPLOT 10126] ojr»eds-Knunoo Jo sierpno Jo uonnq:nsiq c€'p AQEL 


4.2 Results of Modeling Country-Specific Differential Item Functioning 73 


Table 4.33 Relation between 
residuals under the 
generalized partial credit 


Component Correlation Kappa classification CDIF 


Size middle group 


model (GPCM) and 3397  |409» |80 %° 
country-specific factor 1 0.228 0.15 0.20 0.24 
loadings in the bi-factor 2 0.603 0.21 0.29 0.27 
SEN 3 —0.044 0.07 0.17 0.10 
4 0.651 0.46 0.41 0.41 
5 0.519 0.34 0.31 0.25 


Note Correlation between the GPCM residuals and the 
country-specific factor loadings, over countries and items. The 
content of the components is described in Table 3.2. Size middle 
group indicates the classification of the ordered residuals and 
country-specific factor loadings in three categories according to 
their size: a category with negative values, a category with pos- 
itive values and a middle category 

Norms for Kappa: poor agreement = 0.00-0.19, fair 
agreement = 0.20-0.39, and moderate agreement = 0.40-0.59 
"Three equally-sized categories 

"The middle category contained 40 % of the values, the two 
extreme categories each contained 20 % 

"The middle category contained 80 % of the values, the two 
extreme categories each contained 10 % 


(Table 4.21). Regarding help with homework (component 2), Malta was identified 
as having the highest number of outliers in country-specific factor loadings 
(Table 4.28), while The Netherlands, which we earlier identified as exhibiting CDIF 
for component 2 (Table 4.22), also had a high number of outliers. For component 3, 
counting the number of outliers provided little information, as only three outliers 
were counted in the 2.5 96 region (Table 4.29). Hungary did show a high mean 
absolute country-specific factor loading on this component, though the questionable 
reliability of the scale must be kept in mind. Student perception of parental 
involvement (component 4) was measured with the least CDIF in Denmark, 
whereas the school practices on parental involvement from the school perspective 
showed the least CDIF for Italy (Tables 4.30 and 4.31). 

Aggregating the items over the countries provides a tool for further investigation 
of items (Table 4.32), with the same caveats as before; if the 2.5 and 5 96 most 
extreme values are considered, then similarly 2.5 and 5 96 of the residuals must fall 
in this region, but this does not imply that 2.5 and 5 96 of the items have CDIF. No 
item count is prominent. Item 5 (“talk about things you had done") in component 1 
did seem more susceptible to CDIF than other items, since this item revealed the 
greatest number of outliers in country-specific factor loadings over countries. 

We then addressed whether the residual analyses using the GPCM and the 
bi-factor GPCM analyses led to the same conclusions (see Table 4.33). A priori, 
this would be unexpected. The residual analyses target so-called uniform CDIF, 
namely a shift in the item location (item intercept) parameters over countries. The 
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Table 4.34 Correlation and rank correlation between country means estimated with no, 10 and 
20 % cultural differential item functioning (CDIF) parameters, and random item parameters 


Component | Parameter Correlation Rank correlation 

No CDIF | 10 % CDIF | 20 % CDIF | No CDIF | 10 96 CDIF | 20 % CDIF 

1 10 % CDIF - 
20 96 CDIF = 
Random 0.97 

2 10 % CDIF = 
20 % CDIF = 
Random 0.95 

3 10 % CDIF = 
20 96 CDIF = 
Random 0.63 

4 10 % CDIF = 


Random 0.50 0.44 0.37 0.94 0.92 0.89 
5 10 % CDIF | 0.97 = e 0.97 = - 

20 % CDIF | 0.97 1.00 = 0.97 1.00 = 

Random 0.97 0.98 0.98 0.97 0.99 0.99 


Note The components are described in Table 3.2 


bi-factor analyses target non-uniform CDIF, namely differences in the slopes and 
the dimensionality across items. The correlations for components 2, 4 and 5 were 
moderate, while for component 1, the correlation was much lower, and for com- 
ponent 3, the correlation completely vanished. The result for component 3 is 
probably because both the residuals and the country-specific factor loadings are 
poorly estimated for a test containing only three items. 

Though the correlation between the residuals and the country-specific factor 
loadings is a reasonable estimate between the two measures, it does not properly 
indicate to what extent the two measures have the same outliers. To investigate this, 
we ordered and classified the residuals and country-specific factor loadings in three 
categories according to their size (a category with negative values, a category with 
positive values and a middle category). Further, we varied the definition of what 
constituted an outlying value by varying the size of the middle group (assigning it 
variously as 33, 40, or 80 % of values). The calculation of Kappa establishes the 
agreement in categorization between the residual analyses using the GPCM and the 
bi-factor GPCM. This revealed that agreement was poor throughout for component 
3, while, for component 1, the agreement was poor in the 33 % category; for other 
categories in component 1 the agreement was only fair to moderate. In general, the 
results indicate that it is not a good policy to rely on one approach for the inves- 
tigation of CDIF. 

We investigated the influence of CDIF by calculating the correlation and rank 
correlation between country means estimated with no, 10, and 20 % CDIF 
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parameters, and with random item parameters (Table 4.34). Estimates of the means 
using the unidimensional GPCM without country-specific item parameters and 
using the bi-factor GPCM could not be distinguished, so we exclude them from 
further discussion. In general, correlations were high, indicating that, in the esti- 
mation of the country means and the rank order of the country means, CDIF had 
little impact. Component 3 remained the exception; both correlations and rank 
correlations were low. Further, for components 2 and 4, the correlations between 
the means estimated using the GPCM with random item parameters and the other 
three models were also low; however this was not the case for the rank correlations. 
This is because the relationship between means is not linear. We discuss the pos- 
sible influence of CDIF further in the next chapter. 
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Chapter 5 


Relation Between Parental Involvement 
and Student Achievement in PIRLS-2011 


5.1 Method and Rationale for Latent Regression Models 


After modeling parental involvement, we investigated its relationship with reading 
literacy using a three-level regression model in which students (level 1) were 
clustered within schools (level 2) and schools were clustered within countries 
(level 3). Although it is important to recognize that the countries participating in 
PIRLS-2011 cannot necessarily be regarded as being representative of the whole 
world, incorporating a country level in our analyses provides some indication 
whether the parental involvement influences reading achievement in countries 
worldwide. The majority of previous studies on parental involvement applied only 
to the USA. 

In our multilevel model, the dependent variable was the reading literacy variable 
from the PIRLS dataset. To account for the unreliability of this outcome variable, 
five plausible values are available in the PIRLS dataset. All analyses were repeated 
for all five plausible values and then aggregated to overall estimates of fixed and 
random factors, thus incorporating the differences in standard errors for the different 
effect sizes (Von Davier et al. 2009). The sampling procedure was accounted for by 
including a student-class weight at the student level and a school weight at the 
school level. It is important to concurrently use school and student weights in the 
analyses, because schools were sampled first and then students were sampled within 
schools. 

To assess the relationships between the components of interest and the reading 
literacy outcome, important determinants of student achievement such as gender 
and socioeconomic status (SES) were also included. The analytic framework 
described in Chap. 3 included questions that could be regarded as proxies for SES, 
such as “number of books in the home” and “highest educational level attained by 
one of the parents”. Together with gender, these variables were added as control 
variables. 
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We evaluated the different multilevel models for each set of estimates resulting 
from the measurement models described in Chap. 4. Having already compared the 
outcomes for the means on the latent scales for the different corrections for CDIF, 
comparing the results of the structural multilevel model reveals the extent to which 
CDIF may influence the relationship between parental involvement and students’ 
reading literacy, and level of control provided by the measurement models. 

The five variables for the five parental involvement components were generated 
using the five IRT models considered in the previous chapter: the GPCM, the 
GPCM with 10 and 20 % country-by-item interaction, the GPCM with random item 
parameters and the bi-factor GPCM. We included the last four models to assess 
whether taking potential CDIF into account would direct toward different conclu- 
sions. For all five models, we obtained a posteriori estimates for the student 
parameters and entered these estimates as independent variables into the multilevel 
model. 

For the GPCM, first an empty model (model 0) was estimated to see how the 
variance in the outcome variable is distributed over the three levels. Subsequently, 
control variables (student background characteristics) were added as fixed effects 
(model 1). The resulting model can be seen as a baseline to which the models, 
including the parental involvement variables of interest, can be compared. The 
separate parental involvement components were added as fixed effects on either the 
student level (i.e., components 1—4) or school level (component 5), resulting in 
models 2A-2E. We also created a model that included all five components 
simultaneously (model 3). 

By entering the five components as fixed factors, the factor was assumed to have 
the same effect across all countries. However, in the context of this study, we also 
wanted to determine the extent of differences in the effects of parental involvement 
across countries. Therefore, we also considered a model with random slopes at the 
country level for the parental involvement components (model 4). A random slopes 
model includes a variance component for the slope of one or more predictor 
variables, while the other models may be considered special versions obtained by 
fixing parameters. The full model, a random intercepts-and-slopes model, with one 
component for parental involvement is given by: 


Yi = Ban + Bi Gender + p Books + B3Educ + B45, Construct Zem, 


where cu is a normally distributed error term with variance VAR (ei) = g? that is 
independent over students i, schools j, and countries k. The first term on the 
right-hand side is a random intercept, decomposed as 


Boje = Yooo + Uojk + Vk, 


where yooo is the grand mean, and the other two terms are independent normally 
distributed error components with mean zero and variance VAR (ugjx) = & and 
VAR(vk) = tó. The regression coefficients H. ĝa, and B, pertain to gender, the 
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number of books in the home, and the highest educational level attained by one of 
the parents, respectively. The regression coefficient for the parental involvement 
component is decomposed as 


Bu = V400 + Wan + Uae, 


where 7499 is the average slope for the component over all countries and schools, 
and the two error terms have variances VAR (uajt) = E and VAR(u) = 73, 
respectively. Finally, random intercepts and random slopes are allowed to covary; 
at the country level this leads to a parameter COV (vx, Wal = Ty. 

The fixed effects models are obtained by setting VAR(u) = zz = 0, and the 
baseline models, model 0 and model 1, are obtained by removing the appropriate 
predictors. 

To keep the model interpretable and relevant, only the components showing a 
meaningful effect in the fixed model were entered as covariates in the 
random-intercepts-and-slopes model. 

We conducted all analyses using the software package Mplus version 7.11 
(Muthén and Muthén 1998-2012). 


5.2 Results of Latent Regression Models 


The effects on student reading literacy were first modelled using the GPCM without 
correction for CDIF (Tables 5.1, 5.2, 5.3 and 5.4). Model 0 indicates that most of 
the variance in student achievement in reading literacy was situated at the student 
level (44 %; Table 5.1). Differences between countries were also considerable; 
39 % of the variance could be accounted for by between-country differences. This 
was to be expected based on the large range of average country scores reported in 
the international report of PIRLS-2011 (Mullis et al. 2012). 

As expected, model 1 indicated that gender and the two SES-indicators are 
important predictors of reading literacy. On average, girls outperformed boys by 
almost 13 points on the PIRLS-test. The number of books at home and the edu- 
cational level of the parents are both positively related to reading achievement. The 
three background variables explain a considerable amount of variance; 40 % at 
school level and 62 % at country level. This suggests that a substantial part of the 
differences in achievement scores between PIRLS countries can be attributed to 
individual differences in student background characteristics. 

In models 2A—2E, we explored the fixed effects of the different components of 
parental involvement, taken into account the effect of the three background vari- 
ables (Table 5.2). The effect sizes of the background variables did not change 
noteworthy when the different components of parental involvement were included 
in the model. Parental report of literacy activities before their child starts in first 
grade, and helping with homework were both related to a student’s reading literacy, 
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Table 5.1 Effects of student background characteristics and components of parental involvement 
on reading literacy achievement of grade 4 students in 41 PIRLS countries, using a random 
intercept model, without correction for cultural differences 


Effects Model 0 Model 1 

Empty model Student background 

characteristics 

Effect SE Effect SE 
Fixed effects 
Intercept Yooo 525.49 7.12 1541.75 6.30 
Male difference fj, —12.83 0.85 
Books at home (low-high) B; 8.55 0.49 
Parental education (high-low) f; —13.50 0.77 


Random effects 


Variance between students o? 4305.31 (44 96) 168.27 3878.90 (61 96) 165.14 


Variance between schools e 1658.19 (17%) | 269.24 1001.59 (16 96) 177.31 
Variance between countries 1 3887.53 (39 96) 0.30 1497.43 (23 96) 327.48 
Explained by predictors 

At student level 10 % 

At school level 40 % 

At country level 62 96 


but each in a different way. Students of parents reporting spending more time on 
early literacy activities with their child showed higher achievement levels than 
those whose parents spent less time on these activities (the scale runs from high 
involvement to low involvement, therefore the effect in Table 5.2 appears as a 
negative score). This is in agreement with the results presented in the international 
report (Mullis et al. 2012). With regard to helping with homework, there is a 
negative relationship (this scale also runs from high involvement to low involve- 
ment, therefore the effect in Table 5.2 appears as a positive). Because of the large 
number of respondents recorded in the data (over 200,000 students), each rela- 
tionship with achievement, even when very weak, is significant. Therefore, the 
relevance of these relationships was assessed in terms of changes in the achieve- 
ment score if the predictor increased by one standard deviation. The standard 
deviation of early literacy activities was almost 1 (0.97, and thus excluded from this 
report). If parents’ perceptions of the time they spent on early literacy activities 
increased one point Oe, from average to one standard deviation above the mean), 
the score of the student on the PIRLS test increased by nine points. On a scale with 
a mean of 500 and a standard deviation of 100, this could be considered a small 
effect. This also applies to the negative association between helping with homework 
and reading achievement. The standard deviation of this component was also 1, and 
the reduction in student achievement was 9.3 if parents reported that they spent one 
standard deviation more in time in helping their child. 
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Table 5.3 Effects of student background characteristics and parental involvement on reading 
literacy achievement of grade 4 students in 41 PIRLS countries, random intercept model, without 
correction for cultural differences 


Effects Model 0 Model 1 Model 3 

Empty model Student Parental 

background involvement 
characteristics 

Effect SE Effect SE Effect SE 
Fixed effects 
Intercept Yooo 525.49 7.12 541.75 6.30 541.82 5.46 
Male difference P, —12.83 0.85 WIER 0.81 
Books at home 8.55 0.49 6.60 0.48 
(low-high) f; 
Parental education —13.50 0.77 12:15 0.70 
(high-low) f 
Early literacy activities —12.91 0.56 
before primary school 
(high-low) 7409 
Help with homework 13.08 1.14 
(high-low) 500 
Parent’s view of school 4.12 1.50 
practices on parental 
involvement 
(high-low) 7609 
Student’s perception of —4.09 0.49 
parental involvement 
(high-low) 7çoo 
School's perception of 2.7 0.75 


parental involvement 
(high-low) Zeng 


Random effects 


Variance between 4305.31 168.27 | 3878.90 165.14 | 3675.41 163.91 
students o? (44 %) 
Variance between 1658.19 269.24 1001.59 177.31 888.24 153.69 
schools & (17 %) 
Variance between 3887.53 0.30 1497.43 327.48 1190.78 346.23 
countries t (39 %) 


Explained variance by predictors 


At student level 11 % 15 % 
At school level 41 96 46 96 
At country level 64 % 69 % 
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Table 5.4 Effects of student background characteristics and parental involvement on reading 
literacy achievement of grade 4 students in 41 PIRLS countries, random slopes model, without 
correction for cultural differences 


Effects 


Fixed effects 


Component 1 


Component 2 


Component 1 + 2 


Early literacy 


Estimate 


activities 


Help with 
homework 


Estimate 


Estimate 


Intercept Yooo 542.01 6.42 538.54 5.99 538.67 5.78 
Male difference fj, -11.72 0.84 —12.52 0.84 —10.93 0.83 
Books at home 7.19 0.51 8.45 0.45 6.58 0.45 


(low-high) f; 


Parental education —12.76 0.72 —12.96 0.78 —11.97 0.72 
(high-low) f3 

Early literacy —8.67 0.57 —12.66 0.68 
activities 7400 

Help with homework 11.75 1.38 15.15 1.40 
500 

Random effects 

Students c? 3789.18 16003 |3741.12 171.88 | 3618.13 16.048 
Variance intercepts 960.26 170.61 979.91 180.05 930.05 171.372 


schools & 


Variance slopes 33.62 6.04 30.86 5.57 
schools & 

Variance slopes 33.47 6.79 22.74 4.86 
schools é 

Variance intercepts 1386.39 292.43 | 1402.06 321.18 1447.48 371.59 
countries 72 

Variance slopes 14.57 3.80 20.09 5.59 
countries tj 

Covariance intercepts 27.25 28.10 41.40 30.56 
and slopes 12, 

Variance slopes 74.42 14.32 73.96 14.25 
countries t2 

Covariance intercepts 109.23 47.79 134.36 48.19 


2 
and slopes 195 


At first glance, it seems that students whose parents had less positive views 
about school practices outperformed the classmates whose parents held more 
positive views. However, as the standard deviation of this scale was 1.6 and the 
effect size 5.1, the increase in scores was only three points. The same was true for 
students” perception of parental involvement (decrease of almost three points) and 
school perception of parental involvement (increase of three points). These are very 


small effects. 
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Models 2A-2E revealed that for each of the five components, the percentages 
suggested hardly any alteration in the variance, as compared with model 1. Thus, in 
model 3, we entered the fixed effects for all components of parental involvement 
simultaneously (Table 5.3). The influence of early literacy activities and helping 
with homework increased slightly when the effects of the other components were 
held constant; variance increased to 15 % at student level, 46 % at school level, and 
69 % at country level. 

The next step was to estimate two models with random slopes at country level 
for early literacy activities and helping with homework, to determine whether the 
effects of components of parental involvement differed across countries. While 
recognizing this is still open for discussion, we considered these two components as 
showing a small, but meaningful relation with reading achievement. We included 
the three background variables as fixed effects in the random model (Table 5.4). 

For early literacy activities we see a very small increase in the average overall 
effect, from —9.0 in model 2A to —8.7 in model 4. The variance over countries is 
14.57; relative to the total variance in the outcome variable this is very small, but 
relative to the effect of early literacy activities, the effect is clearly larger. 
Ninety-five percent of the range of the slope over countries lay roughly between 
—37.5 and 20.5. A covariance of 27.25 indicated there was a relationship between 
the intercept of a country and the steepness of the slope within a country. A positive 
covariance means that the relationship between parental involvement and reading 
achievement is stronger in countries that performed strongly in the PIRLS test; a 
negative covariance means that the association between the predictor and dependent 
variable becomes stronger as the country average of reading achievement decreases. 
The standard error of the covariance for early literacy activities was larger than the 
covariance, indicating that there was no relation between the intercept and slope. 
From the variance components and the covariance, we obtained a correlation of 
0.07, which must be considered small. 

For helping with homework, the average effect size increased from 9.3 to 11.8. 
The variance of the slope over countries was 74.42; 95 96 of the range of the slope 
over countries lay between —134.0 and 159.5, which can be considered substantial. 
Further, a positive covariance of 109.23 led to a correlation of 0.34, which is also 
substantial. As this scale runs from high involvement to low involvement, although 
the effect reported was positive, in truth it is a negative effect (more help = lower 
achievement). The positive covariance suggests that this negative association of 
helping with homework with achievement was stronger in high-performing 
countries. 

To assess the impact of CDIF, we replicated the last analysis (Table 5.4) with the 
a posteriori estimates of the latent student parameters from all five IRT models 
(Table 5.5). Estimates from all models were very close and never more than one 
standard deviation away from the estimates under the GPCM. We conclude that 
CDIF did not bias the inferences. 
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Table 5.5 Random-intercepts-and-slopes model for effects of student background characteristics 
and parental involvement on reading literacy achievement, without and with correction for cultural 


differences 
Effects GPCM GPCM GPCM Random Bi-factor 
10 % 20 96 GPCM GPCM 
Split Split 
Estimate SE Estimate Estimate Estimate Estimate 


Fixed effects 


Intercept Yooo 538.67 5.78 539.01 539.31 539.39 538.27 
Male difference —10.93 0.83 WIER —10.90 —10.92 —11.11 
By 

Books at home 6.58 0.45 6.59 6.58 6.74 
(low-high) f; 

Parental —11.97 0.72 —11.99 —11.98 —12.10 
education 

(high-low) f 

Early literacy —12.66 0.68 —12.58 —12.56 —12.52 —12.106 
activities 7409 

Help with 15.15 1.40 15.05 15.51 15.59 13.084 
homework soo 

Random effects 

Students o) 3618.13 16.048 | 3619.85 3615.16 3621.08 3628.78 
Variance 930.05 171.372 927.67 928.90 928.49 931.63 
intercepts 

schools & 

Variance slopes 30.86 5.57 30.36 30.91 29.84 29.07 


schools E 


Variance slopes 22.74 4.86 25.34 26.09 24.91 23.49 
schools & 

Variance 1447.48 371.59 1469.70 1418.14 1463.75 1347.119 
intercepts 

countries Ge 

Variance slopes 20.09 5.59 20.47 20.52 21.49 25.418 
countries 14 

Covariance 41.40 30.56 40.32 32.40 40.43 21.73 
intercepts and 

slopes t2, 

Variance slopes 73.96 14.25 72.36 82.66 85.51 47.521 
countries d 

Covariance 134.36 48.19 138.30 132.56 145.50 101.94 


intercepts and 
slopes zs 
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Chapter 6 
Discussion and Conclusions 


The main purpose of this research was to develop a psychometric framework to 
assess the relationship between parental involvement and reading literacy. The 
framework incorporates country specific differences, both at the item level and the 
scale level, to gain insight into cultural differences in the parental involvement 
component and its relation to student achievement in reading literacy. We con- 
ducted secondary analyses on the PIRLS-2011 data of 41 countries. A review of the 
research literature distinguished four dimensions of parental involvement: 
(1) home-based involvement from a parent perspective; (2) school-based involve- 
ment and home-school communication from a parent perspective; (3) home-based 
involvement from a student perspective; and (4) school-based involvement and 
home-school communication from a school perspective. Based on items available in 
the PIRLS data, the first dimension was split in two components: early literacy 
activities and helping with homework. IRT analyses provided item-by-country 
interactions indicating CDIF. The five components were first modeled using the 
unidimensional GPCM. Using these analyses, potential items with CDIF were 
identified and subsequently modeled using country-specific parameters for the 10 
and 20 % most extreme interactions. These methods for identifying and modeling 
CDIF were compared with two other models. The first was the GPCM with random 
item parameters, where the variance of the parameters across countries provided an 
indication of possible CDIF. The second was a bi-factor GPCM where a 
country-specific covariance matrix gave an indication of the extent to which the 
scale loaded on the intended latent variable and the extent of loading on a 
country-specific dimension. Finally, multilevel analyses were conducted to explore 
the association between parental involvement and student achievement for all 
countries that participated in PIRLS-2011. A three-level (student, school and 
country) random intercept model was explored, as well as a random three-level 
model. 
This study addressed three central research questions. 


(1) Which dimensions of parental involvement can be discerned and to what 
extent is there empirical evidence that these dimensions are related to student 
attainment? 
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From the literature review, we constructed a framework of four dimensions that 
combined the different perspectives from which parental involvement can be per- 
ceived De, parent, student and school perspectives) with frequently-mentioned 
dimensions of the construct, such as home-based involvement, school-based 
involvement, and home-school communication. The literature review and analysis 
of the meta-studies in particular, further indicated an overall positive effect of 
parental involvement on student achievement, but considering the individual 
dimensions separately leads to more variable interpretations. For example, the 
relation between helping with homework and student achievement is positive in 
some studies, but non-existent or negative in others (Hoover-Dempsey et al. 2001; 
Patall et al. 2008). These contrasting results were explained by the complexity of 
measurement and lack of agreement between scholars in measuring (the dimensions 
of) parental involvement and the use of single-source data. None of the meta-studies 
recognized other possible reasons for the variable results, such as cultural differ- 
ences in how parents perceive parental involvement or cultural differences in their 
attitude towards their child’s education. In this study, we wanted to examine 
whether these cultural differences could be identified and, if so, whether controlling 
for these differences revealed new information regarding the association between 
components of parental involvement and student’s reading literacy. 


(2) To what extent are there any cultural differences (differences between coun- 
tries) in the components that measure dimensions of parental involvement? 


We developed tools for the identification and modeling of CDIF that were based on 
five models: the GPCM, the GPCM with 10 and 20 % country-specific parameters, 
GPCM with random item parameters and the bi-factor GPCM. Firstly, we found 
that all models clearly and consistently supported the identification of CDIF. 
However, we also found the results obtained by the models varied. There was 
reasonable agreement for components 2 (helping with homework), 4 (student’s 
perception of parental involvement) and 5 (school practices for parental involve- 
ment from a school perspective). The methods clearly disagreed for component 1 
(early literacy activities) and for component 3 (school practices on parental 
involvement, parent perspective); the latter was likely because of the poor reliability 
of this component, probably due to the shortness of the instrument. Disagreement in 
the other four tests is because different aspects of model fit are assessed by the 
models. In fact, the method using residuals specifically targets uniform CDIF, while 
the bi-factor GPCM specifically targets non-uniform CDIF. In conclusion, practi- 
tioners should not rely on one model and one approach to investigate CDIF, but 
diversify in their methods. 

Finally, and most importantly, analyzing the influence of CDIF on the estimates 
of country means and on the outcomes of latent regression analyses led to the 
conclusion that CDIF did not influence the results. Considering all the differing 
components of parental involvement, CDIF had no influence on these items within 
the PIRLS survey. 
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(3) To what extent are the different dimensions of parental involvement related to 
student achievement in reading literacy, taking into account student back- 
ground characteristics and differences between countries? 


The results of the three-level models with a random intercept showed that, con- 
trolled for student’s gender and SES, and taking into account between-schools and 
between-countries variance, there is a rather weak but positive relationship between 
early literacy activities and student achievement in reading literacy at grade 4. This 
positive association supports the Dutch study of Kloosterman et al. (2011), who 
also found that early literacy activities were positively related to student reading 
achievement at primary school. We may here only confirm a positive association 
and cannot make any claims about causality, as PIRLS is cross-sectional. The 
results only indicate that other types of studies (experimental studies) measuring the 
real effects of early literacy activities on reading achievement are relevant, assuming 
that there is agreement among scholars in how these activities should be measured. 
Our analyses have shown that if written questionnaires are applied, the current 
PIRLS scale seems to work identically in a large number of countries and cultures. 
Further, the reliability of the first two components, early literacy activities and help 
with homework, meets the minimum standard for a survey of 0.70 within all 
countries. The scale for parental involvement from the school perspective often met 
this standard. The scale for parental involvement from the student perspective did 
not meet this standard, though it consisted of 15 items. Component 3 (school 
practices on parental involvement, parent perspective) turned out to be an unreliable 
scale, probably because it contained only three items. There is clearly margin for 
improvement in these last three scales. 

The results of two meta-studies on homework involvement and its relation with 
student attainment were inconclusive (Hoover-Dempsey et al. 2001; Patall et al. 
2008). The exploration of the PIRLS data revealed helping with homework had a 
small negative effect. This may be explained by the so-called reactive hypothesis, 
suggesting that parents tend to react with a higher level of involvement if their child 
is falling behind at school (McNeal Jr. 2012). McNeal Jr. (2012) and Cooper et al. 
(2000) suggested another explanation for this negative association; helping with 
homework might also interfere with learning if parents are not sufficiently equipped 
to help, if they are too eager (which affects the self-confidence of their child), or if 
their instruction is very different from the instruction of the teacher. Again, based on 
the PIRLS-2011 data, it is difficult to analyze how helping with homework affects 
student reading literacy, but further exploration of the indirect effect of helping with 
home via some measure of student self-confidence in reading would be useful. 
Another suggestion for future PIRLS studies would be to ask parents how confident 
they feel about helping their child with homework and whether they feel sufficiently 
informed about how reading is taught at school. 

Both early literacy activities and helping with homework are home-based 
activities, confirming that what parents do at home with their child is important for 
student achievement. In this study we found school-based involvement from the 
perspective of the school (component 5) had negligible effect. As the constructed 
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scale for school-based involvement from the perspective of the parent (component 
3) turned out to be unreliable, we are unable to draw valid conclusions for this 
component regarding its relation with student achievement. 

Overall, the impact of parental involvement on reading literacy is not large. 
When all five components were entered into the model, it explained approximately 
15 % at the student, 46 % at the school level, and 69 % at the country level. 
However, the impact differences across countries proved to be quite large, espe- 
cially for helping with homework, where regression coefficients, with a mean value 
of 11.8, range over countries from —134.0 to 159.5. Finally, the country-level 
intercept and slopes for helping with homework have a substantial positive corre- 
lation of 0.34. In low-achieving PIRLS countries, the effect of helping with 
homework is smaller than in high-performing countries. This means that, in 
exploring the achievement effect of helping with homework, the educational con- 
text should be taken into account. The sometimes contradictory results of earlier 
studies on this subject (Hoover-Dempsey et al. 2001) may also be explained by 
such differing effects between countries. 

Another explanation for the positive correlation between intercepts and slopes on 
the country level would be that, in low-achieving countries, parents’ reading 
competency will also be low, so parents are themselves less able to read and hence 
provide effective support. However, it is beyond the possibilities of the present 
research to draw conclusions in this respect. 

In PIRLS, the literacy test and background questionnaires are translated and 
adapted for each country. Considerable effort is devoted to guaranteeing the 
international validity of these instruments. For example, the translations and the 
layout of the instruments are thoroughly reviewed by independent verifiers, and all 
necessary adaptations are documented in detail. However, it is not unlikely that 
there are cultural differences in the way respondents interpreted some of the 
questionnaire items. The main purpose of this study was to establish whether there 
were any cultural differences in the measurement of parental involvement in PIRLS 
and, if so, whether correction for these differences led to different results with 
regard to its relation with reading literacy. Although some of the PIRLS scales for 
parental involvement may require improvements to increase their reliability, the 
overall conclusion is that these scales are internationally valid. 
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Appendix A 
Technical Details on the Implementation 
of the Bi-factor Model 


The estimation equations and expressions for the covariance matrix of the estimates 
are easily derived using Fisher's identity (Efron 1977; Louis 1982; Glas 1999). The 
identity plays an important role in the framework of the EM algorithm, which is an 
algorithm for finding the maximum of a likelihood marginalized over unobserved 
data. The principle can be summarized as follows. Let Lo(2) be the log-likelihood 
function of parameters À given observed data xo, and let L(A) be the log-likelihood 
function given both observed data xo and unobserved missing data x,,. The latter is 
called the complete data log-likelihood. The interest is in finding expressions for the 
first-order derivatives of Lo(A), say, the expressions for L'o(4). Define the first-order 
derivatives with respect to the complete data log-likelihood as L',,(4). Then Fisher's 
identity entails that L’o(A) is equal to the expectation of L’,,(A) with respect to the 
posterior distribution of the missing data given the observed data, p(x,,|xo; A), that 
is, 


Dt = EI, 4) = J F Dle a ba, 


To apply this framework to IRT, a very general definition of an IRT model is 
adopted. Assume an IRT model is defined by the probability of a response pattern 
Xn, Which is a function of a, possibly vector-valued, student parameters 0,,, and item 
parameters a and b, which are item discrimination and item location parameters of 
an IRT model. So the IRT model is given by p(x, | 0,, a, b). Assume further that the 
student parameter 0, has a normal density N(8,; soy Xeon) where, again, g(n) is 
the country to which student n belongs. The key idea is to view the student 
parameters @,, as missing data and the item and population parameters a, b, Mein), 
and Za as structural parameters 1 to be estimated. Then the complete data 
log-likelihood for a student n is 
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Len) (A) = log p( Xl 0,0; Ong, a, b) + log N(O,o, Ong; Hg(n)> Lo(n)) 


and so the estimation equations are given by 


Lo (2 mi 
OLo(A) E YE Ono NET — Dol, X4,4|] = 0 
n j=l 


aio 
D = Kä E| Ong sey — Doll le, Alz 0 
dig n|g(n)=g = 
Ota) | 
ed; 7 2 [E (Pin) fin, 4) — Xnij] =0 
PE _ 5 Hg — E(0,0|x,, 2) =0 
Hg n|g(n)=g 


OLo(A) _ y ai ie, ~ iis 2) =" 


nig(n)=8 
where all the expectations are relative to the posterior distribution 
IG OnglXn, 2) x< p( Xa 0,0; Ong, a, b)N(On0, Ong; Hg; X). 


We undertook all calculations using the public domain software package MIRT 
(Glas 2010). The program uses the EM-algorithm to solve the estimation equations 
and Gaussian quadrature to evaluate the integrals. 
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